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Introduction 


EMBEDDED  COMPUTER  RESOURCES 
AND  THE  DSARC  PROCESS 


In  the  Spring  of  1975,  a major  new  initiative  was  undertaken  in  the  area 
of  software  management  for  weapons,  communications,  command  and  control, 
and  intelligence  systems.  Organizational  steps  which  give  the  proper 
leverage  have  been  taken;  OSD  policy  direction  on  the  management  of  com- 
puter resources  has  been  issued;  impacts  are  gradually  being  made  on 
major  systems;  visibility  of  software  as  a major  system  component  and 
decision  parameter  is  being  felt. 

In  order  to  hasten  the  assimilation  of  sound  software  management  policies, 
practices,  procedures,  and  technology  into  the  normal  system  acquisition 
and  review  process  as  practiced  by  OSD  and  the  Military  Departments,  a 
guidebook  covering  salient  embedded  computer  resource  issues  in  the  con- 
text of  the  Defense  Systems  acquisition  Review  Council  (DSARC)  process 
has  been  assembled.  This  guidebook,  provided  in  the  attached  paper,  is 
organized  into  the  following  sections: 

Part  I General  DSARC  Issues  (DSARC  I. II. Ill) 


Part  III 


Part  IV 


Part  V 


Part  I General  DSARC  Issues  (DSARC  I, II ,111) 

Part  II  Sensitivity  Data  and  Relationships 

Part  III  Program  Inquiry  Questions 

Part  IV  Software  Cost  Estimation  Guidelines 

Part  V List  of  Reference  Documents  - Policy,  Practice,  Procedure 

\ 

Part  I of  The  *^uidebook  reviews  the  purpose  and  scope  of  each  DSARC 
^milestone,  and  translates  this  into  specific  questions  and  issues  appro- 
priate for  the  computer  resoqrce  and  software  product.  Each  question  or 
issue  is  followed  by  t1)e  spectrum  of  responses  typically  >eceiVfftf..>  When- 
ever possible,  responses  tiere  ranked  in  increasing  order  of  preference 
enabling  the  guidebook  user  to  develop  his  own  /’figure  of  merif^as  a 
function  of  the  importance  of  any  issue  (or  set  of  issues)  to  the  program 
under  review.  - 
/ 


{ 


Part  II  presents  quantitative  sensitivity  data  for  twenty-four  managerial 
and  technical  factors  which  relate  directly  to  development  productivity, 
and  hence  to  cost,  schedule,  and  performance  risk.  These  parametric 
relationships  are  useful  in  assessing  and  forecasting  the  implications  of 
management  and  technical  decision  options.  Part  III  of  the  guidebook 
provides  a set  of  inquiries*  which  although  too  detailed  for  DSARC 
itself  ,$sT'a~usefui  tool  in 'assessing  the  overall  1,jiea}th*’'  of  a software 
development  effort  at  any  given  point  in  time.  Many  of  the  questions 
in  this  part  will  not  have  answers  early  in  a program  (i.e.,  prio^  to 
or  shortly  after  DSARC  I)  but  as  progress  is  made,  a larger  percentage  of 
them  will  indeed  be  answerable.  If  at  DSARC  III,  a significant  percentage 
(i.e.,  greater  than  10-15%)  remain  with  vague  or  non-existent  responses, 
there  may  be  cause  for  serious  concern  about  the  readiness  for  a 
favorable  production  decision.  Merely  reviewing  the  responses  (or  lack 
thereof)  to  these  inquiries  will  provide  a good  sense  of  development 
progress  and  the  rate  thereof,  and  will  aid  in  the  determination  of 
readiness  to  proceed  into  the  next  major  system  phase. 

Part  IV  of  the  guidebook  gives  an  empirically  derived  relationship  for 
checking  and  verifying  the  reasonableness  of  program  manager  supplied 
software  cost  and  size  estimates.  Part  V is.  a list  of  the  pertinent 
policy,  practice  and  procedure  documents  which  govern  embedded  computer 
resources  and  the  DSARC  process,.’  Appendix  A contains  relevant  sections 
of  several  documents  from  Part  V. 

The  material  provided  in  this  guidebook  is  intended  to  assist  OSD 
staff  officers  and  their  counterparts  in  the  Military  Departments  in 
preparing  for,  and  conducting  meaningful  review  of  an  increasingly 
important  component  of  Defense  Systems.  It  is  not  intended  as  a set 
of  inflexible  rules  which  substitute  for  managerial  and  technical 
judgement,  and  should  not  be  used  as  such. 

The  Management  Steering  Committee  - Embedded  Computer  Resources,  and  the 
members  thereto  are  available  to  assist  with  questions,  comments,  and 
implementation  of  material  contained  herein.  Appropriate  names,  loca- 
tions, and  telephone  numbers  are  provided  in  the  Appendix  B to  this 
document. 

Comments  and  suggestions  on  material  contained  herein  should  be  addressed 
to  Barry  C.  De  Roze,  Office  of  the  Secretary  of  Defense,  Room  2A318  of 
the  Pentagon.  The  appropriate  telephone  number  is  (202)  695-0121.* 

C /$&. - 

Barry  C.  De  foze  ^ 

For  the  Management  Steering  Committee  - 

Embedded  Computer  Resources 


* Annual  updates  to  the  material  are  planned.  Comments  and  suggestions 
will  be  held  for  inclusion  in  the  next  edition. 
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PART  1 


GENERAL  DSARC  ISSUES  (DSARC  I,  II,  III) 


The  purpose  of  this  section  is  to  provide  guidelines  to  determine  adequacy  of 
computer  resource  utilization  and  planning.  Questions  and  typical  responses  are 
organized  into  three  sections  based  on  the  three  major  Defense  System  Acquisiton 
Review  Council  (DSARC)  meetings.  (DSARC  meetings  are  normally  required  for 
all  weapon  systems  involving  over  $75  million  in  research,  development,  test  and 
evaluation  or  over  $300  million  in  production.  DSARC  L H,  and  III  meetings  are 
held  prior  to  entering  the  demonstration  and  validation  phase,  the  full-scale 
engineering  development  phase,  and  the  production  and  deployment  phase  respec- 
tively and  are  used  as  input  to  the  Secretary  of  Defense  who  decides  if  the  system 
should  proceed  to  the  next  phase.)  Embedded  computer,  i.e.  those  computers 
integral  to  a weapon  systems  from  a design,  procurement,  and  operations 
viewpoint,  will  be  emphasized  rather  than  general  purpose  computers  that  also  may 
be  used  to  provide  support  for  some  systems.  Note  that  embedded  computers 
range  from  small  units  in  airborne  systems  to  large  units  in  ground  and  ship  based 
command  and  control  systems. 

This  is  not  a "how-to-do-jt"  guide.  Instead,  it  consists  of  a series  of  questions 
that  OSD  personnel  could  ask  prior  to  DSARC  I,  II,  and  III.  After  each  question  are 
several  possible  responses  that  might  be  given  by  the  Program  Manager's  staff. 
When  it  was  possible,  the  responses  were  ranked  in  increasing  order  of  preference. 
By  performing  a more  thorough  review  of  embedded  computer  resources,  reviewing 
personnel  can  help  to  eliminate  many  of  the  problems  that  exist  in  recently 
developed  systems  utilizing  embedded  computers. 


GENERAL  DSARC  ISSUES 


DSARC  ! 


General  I_3 

Operation  Requirements 1-5 

Life  Cycle  Management  1-6 

Tradeoffs  

Use  of  Existing  Hardware  & Software  1-9 

Possible  Future  Problem  Areas  1-10 


DSARC  II 


General  j.^2 

Operational  Requirements  1-13 

Life  Cycle  Management  1-14 

Tradeoffs  

Program  Managers'  Staff  I-l~ 

Project  Control I_18 

Development  Contract 1.20 

Testing  1-21 

Software  Reliability  and  Maintainability  1-24 

Miscellaneous  I_28 


DSARC  D1 


General  1. 29 

Present  Status  I_29 

Life  Cycle  Management  1-32 

Miscellaneous  1-35 
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DSARC  I QUESTIONS  AND  RESPONSES 


The  DSARC  I decision  point  is  reached  when  competitive  exploration  of 
alternative  systems  concepts  has  been  completed  and  selected  alternatives  warrant 
system  demonstration. 


DSARC  1 - General  Issues 

|| 

1.  Has  the  Program  Manager's  Office  been  set  up  and  staffed?  (If  so,  answer 

DSARC  11  questions  on  Program  Manager's  staff.) 

A.  No;  the  program  is  being  handled  by  a Service  laboratory. 

B.  No;  the  program  is  being  handled  by  a major  Service  command. 

C.  An  office  has  been  established  but  only  minimum  staff  has  been 
obtained. 

D.  Yes;  DSARC  II  questions  on  Program  Manager's  staff  have  been 
answered. 

2.  What  steps  are  being  made  to  insure  software  visibility? 

A.  All  software  items  will  be  listed  as  deliverables. 

B.  Cost  for  software  will  be  a separate  cost  reporting  item. 

C.  Software  will  be  put  under  configuration  control  early  in  the  develop- 
ment process. 

D.  A person  on  the  Program  Manager's  staff  will  be  the  focal  point  for  all 
software  development. 

E.  Government/contractor  design  reviews  will  be  performed  for  software 
as  well  as  hardware. 

F.  Software  risks  have  been  assessed  and  "software  first"  demonstrations 
are  planned  for  all  risk  areas. 

G.  All  of  the  above  steps. 

3.  With  what  other  systems  will  the  system  have  to  interface?  What  knowledge 

of  system  implementation  of  the  external  system  is  required?  b this 

information  available? 

A.  The  system  is  completely  inc.  , 'dent. 
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B.  The  system  will  eventually  transmit  and  receive  data  from  higher 
headquarters;  the  higher  headquarter's  unit  has  not  been  designed  yet. 
Information  on  data  formats  is  required  but  is  not  available. 

C.  The  system  must  interface  with  an  existing  external  guidance  system 
and  a manual  override  system  currently  under  development.  Interfacing 
information  on  both  systems  is  available. 

Will  more  than  one  agency  or  contractor  develop  software  for  the  system?  If 

so,  who  will  arbitrate  disputes  among  them?  Will  the  groups  participate  in 

each  other's  critical  design  reviews? 

A.  The  main  contractor  will  have  many  subcontractors  that  will  develop 
software.  No  disputes  are  expected.  Subcontractors  will  not 
participate  in  each  other's  critical  design  reviews. 

B.  One  contractor  will  develop  software  for  the  ground-based  control 
system  and  another  will  develop  software  for  the  weapon  itself. 
Disputes  between  contractors  will  be  resolved  by  the  Program 
Manager's  staff.  Neither  contractor  will  participate  in  each  other's 
reviews. 

C.  The  main  contractor  will  develop  software  to  be  located  in  the  weapon 
system  and  a subcontractor  will  develop  software  for  the  weapon 
support  system.  The  contractor  will  arbitrate  all  disputes.  Both  will 
attend  each  other's  critical  design  reviews. 

D.  Only  one  contractor  will  develop  software. 

How  will  critical  design  reviews  be  conducted? 

A.  The  contractor  will  explain  the  selected  system  design  to  the  Program 
Manager's  staff;  all  questions  will  be  answered.  After  approval,  work 
will  proceed. 

B.  The  contractor  will  provide  technical  information  one  week  before 
design  reviews.  The  Program  Manager's  staff  will  study  all  material 
before  the  review  meetings.  The  contractor  will  present  the  system 
design  and  answer  all  questions  during  a full  day  review.  After 
approval,  work  will  proceed. 

C.  The  contractor  will  provide  pertinent  technical  information  one  month 
before  design  reviews.  During  the  1 week  review,  alternative  designs 
and  reasons  for  the  nonselection  will  be  reviewed  in  detail.  If 
alternative  designs  that  have  not  been  considered  previously  are 
uncovered,  the  critical  design  review  will  be  rescheduled.  After  the 
Program  Manager's  staff  is  satisfied  that  the  best  design  has  been 
selected,  approval  to  proceed  will  be  granted. 
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DSARC  1 - Operational  Requirements  Issues 


1.  Are  the  system  operational  requirements  well  defined?  Have  they  stabilized? 
Are  they  realistic?  How  can  this  be  proven? 

A.  Many  requirements  will  depend  on  results  of  further  development. 

B.  The  requirements  may  not  be  within  the  state-of-the-art  (this  will  be 
determined  during  demonstration  and  validation).  Requirements  are 
well  defined  and  stabilized;  requirements  have  been  quantified  and  are 
testable. 

C.  The  requirements  are  well  defined  but  have  not  been  frozen. 

D.  The  requirements  are  well  defined  and  have  stabilized.  Attainment  of 
requirements  can  be  easily  measured. 

2.  When  and  how  will  a validation  of  computer  resources  requirements, including 
software,  be  conducted?  What  are  the  software  areas  of  greatest  risk?  How 
will  risk  analysis  be  performed? 

A.  Immediately  after  DSARC  II;  only  proven  hardware  will  be  used  so  no 
risk  analysis  will  be  needed. 

B.  Validation  will  be  made  during  full-scale  engineering  development.  The 
time  to  develop  efficient  input/output  modules  will  be  the  biggest  risk; 
risk  will  be  lowered  by  hiring  more  people. 

C.  Validation  will  be  done  prior  to  DSARC  II  by  using  competitive 

preliminary  development  by  two  different  contractors;  the  best  system 
design  will  be  chosen  based  on  schedule  and  costing  estimations.  The 

main  risk  area  is  computer  timing  due  to  the  large  amount  of 

mathematical  processing  that  will  be  required.  Risk  analysis  will  be 
performed  by  qualitatively  ranking  all  major  risk  areas. 

D.  Validation  will  be  made  during  the  demonstration  and  evaluation  phase 
by  using  a software-first  emulation  technique.  The  time  and  cost  to 
develop  the  executive  program  presents  the  greatest  risk.  A decision 
risk  analysis  computer  program  will  be  used. 

3.  Which  operation  requirements  are  likely  to  change  during  development  of  the 
system?  During  system  deployment?  How  will  the  software  accommodate 
these  changes? 

A.  Since  the  system  is  entirely  independent  of  other  systems,  the 

operational  requirements  will  not  change. 
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B.  Enemy  threats  will  change.  All  parameters  concerning  enemy  systems 
will  be  located  in  special  memory  locations;  changing  those  memory 
locations  will  update  the  system  capability. 

C.  Enemy  countermeasures  will  be  developed  after  the  system  has  been 
deployed.  Software  will  be  modular,  allowing  easy  modification  by  the 
software  support  agency. 

D.  Equipment  with  which  the  weapon  system  must  communicate  will 
change.  Software  will  be  written  so  that  only  the  input  and  output 
modules  must  be  changed. 


DSARC  1 - Life  Cycle  Management  Issues 


1.  Who  will  provide  post  development  support?  Why  was  this  selection  made? 

A.  Unknown  at  this  time. 

B.  The  contractor  who  will  develop  the  system;  development  costs  will  be 
kept  to  a minimum. 

C.  Government  personnel;  follows  support  doctrine  of  the  Military  Depart- 
ment. 

D.  Originally  the  contractor,  later  government  personnel;  this  will  mini- 
mize total  life  cycle  costs. 

2.  When  will  the  system  life  cycle  support  activity  and  the  software  life  cycle 
activity  be  designated? 

A.  After  DSARC  HI. 

B.  After  DSARC  U. 

C.  Both  activities  have  been  designated. 

3.  Will  Operations  and  Maintenance  funds  be  requested  to  support  contractor 
activities  directed  toward  providing  maintenance  capabilities  and  documenta- 
tion? 

A.  No 

B.  Yes 

U.  How  and  when  will  maintenance  provisions  be  specified? 

A.  "Appropriate  maintenance  provisions"  will  be  included  in  the  contract 
for  full-scale  engineering  development. 


B.  Maintenance  provisions  will  be  part  of  the  development  contract;  the 
maintenance  contract  will  include 

3.  How  will  support  requirements  be  determined? 

A.  They  will  be  determined  by  the  Program  Manager's  staff. 

B.  Support  requirements  will  be  determined  by  discussions  with  users  of 
similar  systems  that  are  already  deployed. 

C.  Requirements  will  be  supplied  by  the  appropriate  support  agencies, 
when  they  are  designated. 

D.  A combination  of  elements  of  responses  A,  B,  and  C. 

6.  What  computer  hardware  will  be  unique?  Why  can't  standard  hardware  be 

used  and  how  will  replacement  parts  be  obtained? 

A.  Computer  requirements  are  not  known. 

B.  A unique  processor  is  needed  to  efficiently  utilize  a new  programming 
language  that  has  been  chosen  to  minimize  program  development  time. 
Replacements  will  be  bought  sole  source  from  the  manufacturer. 

C.  A unique  processor  already  embedded  within  the  system  (e.g.  a 
microprocessor  contained  in  the  navigation  equipment)  will  be  expand- 
ed. Compatable  processors  can  be  procured  from  several  manufacturers 
that  sell  the  processor  to  the  commercial  market. 

D.  New  hardware  is  required  because  existing  processors  are  not  fast 
enough.  All  unique  parts  for  the  expected  life  of  the  system  will  be 
purchased  at  one  time  shortly  after  system  deployment. 

E.  No  unique  computer  hardware  will  be  utilized. 


DSARC  I - Tradeoff  Issues 


1.  What  tradeoffs  will  be  made  between  the  embedded  computer  and  other 

methods  of  meeting  the  system  requirements? 

A.  None;  operational  requirements  cannot  be  meet  without  the  embedded 
computer. 

B.  Manpower  and  support  costs  required  to  perform  the  required  task  with 
existing  methods  and  equipment  will  be  compared  with  the  develop- 
ment, manpower,  and  support  costs  of  the  computerized  equipment. 

C.  A simple  system  requiring  a highly  skilled  operator  to  evaluate 
computer  results  will  be  compared  to  an  elaborate  system  to  be  used  by 


a Jess  skilled  operator.  Tradeoffs  will  be  based  on  development  and 
support  costs,  including  operator  training  required. 

2.  How  will  hardware/software  tradeoff  be  made? 

A.  Hardware  will  be  minimized  in  order  to  meet  design-to-cost  goals. 

B.  Maximum  use  of  software  will  allow  greater  flexibility  for  future 
changes. 

C.  Time  critical  tasks  will  be  done  with  hardware;  all  others  with  software. 

D.  Engineers  will  make  tradeoffs  during  the  demonstration  and  evaluation 
phase,  based  on  availability  of  newer  hardware,  changing  needs,  etc. 

3.  How  will  the  processor  architecture  be  determined? 

A.  The  architecture  will  be  determined  by  selecting  the  lowest  cost 
computer. 

B.  We  will  select  the  architecture  that  will  minimize  program  development 
time. 

C.  The  architecture  that  will  minimize  cost  of  computer  and  interface 
electronics  will  be  selected. 

D.  The  architecture  that  will  minimize  time  to  perform  required  calcula- 
tions will  be  selected. 

E.  The  architecture  that  will  minimize  life  cycle  costs  (including  increas- 
ing weapon  system  capabilities  after  deployment)  will  be  chosen. 

U.  How  will  the  processor  memory  capacity  be  determined? 

A.  The  maximum  memory  capacity  will  be  selected. 

B.  Memory  capacity  needs  will  be  extrapolated  from  similar  existing 
equipment. 

C.  The  required  capacity  will  be  determined,  then  a fixed  percentage  will 
be  added  for  future  expansion. 

D.  An  estimate  of  how  requirements  might  change  will  be  made,  then  an 
estimate  of  how  much  additional  memory  capacity  would  be  required  to 
handle  those  changes  will  be  made. 

5.  How  will  excess  memory,  processor  time  and  capability  needs  be  determined? 

A.  The  system  with  largest  memory,  fastest  processing  speed  and  most 
capability  will  be  used. 
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B.  They  will  be  extrapolated  from  similar  systems. 

C.  Modules  containing  critical  timing  have  been  programmed;  timing 
results  were  used  as  a basis  for  estimates. 

6.  How  will  the  team  that  wrote  the  original  requirements  document  be  kept  for 

tradeoff  studies? 

A.  A separate  team  will  do  the  tradeoff  studies. 

B.  The  team  will  be  "on  call"  during  the  demonstration  and  validation 
phase. 

C.  The  team  will  remain  attached  to  the  Program  Manager's  Office  until 
tradeoffs  are  made. 


DSARC  1 - Use  of  Existing  Hardware  and  Software  Issues 


1.  What  new  technology  (computer,  sensor,  and  control)  must  be  used? 

A.  The  sensor  to  be  developed  must  have  an  order  of  magnitude  better 
resolution  to  detect  the  anticipated  stimulus. 

B.  The  computer  must  process  data  faster  than  can  be  done  with  existing 
computers. 

C.  Real-time  control  will  require  development  and  implementation  of  a 
new  control  algorithm. 

D.  None 

2.  What  special  tasks  must  be  performed  in  the  demonstration  and  validation 

phase  to  perfect  new  technologies? 

A.  It  is  not  known  for  certain  at  this  time. 

B.  Special  tasks  will  be  determined  in  joint  contractor -user  discussions 
after  the  demonstration  and  validation  contract  has  been  awarded. 

C.  (Generalized  list  of  tasks  to  be  performed.) 

D.  (Detailed  list  of  tasks  to  be  performed.) 

3.  How  much  system  design  can  be  obtained  "off-the-shelf"  from  previous  sys- 
tems? 

A.  None;  the  weapon  system  being  replaced  had  no  embedded  computer. 
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B.  Little;  the  new  embedded  computer  has  different  architecture  and 
computer  word  length. 

C.  A medium  amount;  the  same  guidance  hardware  will  be  used;  however, 
the  method  of  calculation  of  guidance  parameters  is  different.  The  new 
system  will  use  a similar  operating  system. 

D.  A large  amount  - software  is  a minor  extention  of  a previously 
developed  system. 

Which  existing  operational  application  and  support  packages  will  be  utilized? 

Are  the  application  programs  operational  on  the  target  computer?  If  not, 

what  are  the  major  hardware/software  differences?  To  what  extent  have  the 

contractor's  personnel  used  these  packages  previously? 

A.  Similar  application  programs  have  been  written,  but  for  systems  having 
very  different  architectures  and  input/output  methods  (all  applications 
programs  must  be  rewritten).  Since  no  support  software  is  available,  it 
will  have  to  be  written  by  the  contractor. 

B.  Control  modules  used  in  another  system  will  be  used  as  will  an  extensive 
library  of  support  software.  Application  programs  will  have  to  be 
modified  due  to  different  word  lengths  of  the  original  and  the  target 
computers.  Contractor's  personnel  have  not  used  these  control  or 
software  support  programs  before. 

C.  No  existing  operational  application  programs  exist.  Support  packages 
such  as  cross-assemblers  and  compilers  will  be  used;  contractor's 
personnel  are  familiar  with  these  packages. 

D.  Numerical  processing  modules  previously  developed  for  the  target 
computer  will  be  used.  Support  packages  to  be  utilized  were  developed 
by  the  contractor  for  a previous  system;  most  contractor  personnel  have 
used  them. 


DSARC  I - Possible  Future  Problem  Areas  Issues 


Has  preliminary  systems  analysis  been  performed?  What  hardware  and/or 
software  problems  areas  were  discovered? 

A.  No  systems  analysis  was  needed  or  performed. 

B.  System  analysis  indicated  a possible  saturation  of  available  computation 
time. 

C.  The  interconnection  of  the  many  embedded  computers  within  the 
weapon  system  may  require  more  man-months  than  originally  estimat- 
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D.  System  analysis  indicated  computer  memory  needed  may  require  more 
electrical  power  than  is  available. 

2.  How  will  the  problems  of  question  1 be  handled  in  the  demonstration  and 

validation  phase? 

A.  Since  problems  cannot  be  readily  envisioned,  solutions  will  be  handled 
on  a case-by-case  basis. 

B.  A faster  computer  might  be  needed. 

C.  Extra  space  might  be  needed  within  the  weapon  for  additional  memory 
and  power  supply. 

D.  The  single  embedded  computer  should  be  replaced  with  several 
microprocessors,  each  performing  a dedicated  function. 

3.  What  critical  areas  must  be  resolved  during  the  demonstration  and  validation 

phase?  How? 

A.  Can  computing  system  be  made  small  enough  to  meet  desired  size  and 
weight  restrictions?  Prototype  modules  and  techniques  needed  for 

f * ' manufacturing  will  be  developed  earlier  than  normal. 

B.  Can  design-to-cost  goals  be  met?  Costs  will  be  considered  during  all 
internal  engineering  reviews. 

C.  Can  the  weapon  system  be  maintained  in  the  field?  Limited  field 
testing  will  be  performed  before  full-scale  development  to  determine 
maintainability. 

U.  Do  you  envision  other  risky  areas?  What  are  your  plans  to  resolve  these 

problems? 

A.  Schedules  seem  optimistic  due  to  state-of-the-art  techniques  to  be 
utilized;  schedules  will  be  monitored  closely. 

£.  Interfacing  of  many  embedded  computers  within  the  weapons  system 
will  be  required;  a common  data  bus  for  interconnecting  aii  systems  will 
be  specified. 

C.  Adequate  monitoring  of  software  development;  the  Program  Manager 
will  request  assistance  from  the  Service  command  dealing  with 
software. 

D.  Real-time  testing  of  the  completed  system  will  be  difficult;  testing 
hardware  and  software  will  be  developed  concurrently  with  the  weapon 
system. 
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DSARC  II  QUESTIONS  AND  RESPONSES 


The  DSARC  D decision  point  is  reached  when  the  demonstration  and 
validation  activity  has  been  completed  and  a recommendation  on  the  preferred 
systems  for  full-scale  engineering  development  can  be  made. 


| DSARC  D - General  Issues 

i 

What  are  the  present  problems  and  the  plans  for  resolving  them? 

A.  No  present  problems. 

B.  Minor  problems  only;  "they  will  take  care  of  themselves". 

C.  (List  of  major  problems  and  list  of  "get  well"  actions.) 

How  do  you  know  present  cost  and  time  estimates  are  sound? 

A.  Contractor's  and  Program  Manager's  staff  agree  estimates  are  sound. 

B.  Estimates  were  based  on  a similar  system  developed  for  another  service 
several  years  ago. 

C.  All  estimates  were  developed  by  personnel  with  past  records  of 

accurate  estimates. 

D.  Cost  and  time  estimates  were  developed  from  small  program  elements 
that  could  be  accurately  estimated. 

Of  the  total  computer  resources  to  be  spent  during  full-scale  engineering 
development,  what  percentage  will  be  used  for  design,  for  coding,  and  for 
testing?  What  have  these  percentages  been  for  the  contractor  in  the  past? 

A.  Design  25  percent,  coding  *0  percent,  testing  35  percent.  Past 

contractor  experience  was  not  on  systems  of  comparable  complexity. 

B.  Design  30  percent,  coding  35  percent,  testing  35  percent.  Past 

experience  was  about  one-third  in  each  category. 

C.  Design  40  percent,  coding  20  percent,  testing  40  percent.  Past 

experience  was  design  30  percent,  coding  25  percent,  test  ng  45 
percent. 
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DSARC  II  - Operational  Requirements  Issues 


1.  Have  the  system  operational  requirements  changed  since  DSARC  I?  Are  they 

now  stabilized? 

A-  Requirements  change  as  costs  can  be  calculated;  the  weapon  system 
would  cost  too  much  if  it  had  to  meet  original  requirements. 

B.  Minor  changes  were  needed  due  to  new  enemy  threats;  these  types  of 
changes  are  expected  to  continue  throughout  system  development. 

C.  Major  changes  were  made  due  to  availability  of  improved  hardware; 
requirements  have  remained  stable. 

D.  Requirements  have  not  changed. 

2.  How  will  the  system  design  be  validated  prior  to  implementation? 

A.  Design  is  similar  to  other  systems  that  performed  satisfactorily. 

B.  System  design  followed  directly  from  the  embedded  computer  architec- 
ture; any  other  design  would  not  use  the  computer  efficiently. 

C.  Design  will  be  validated  by  using  simulation. 

D.  System  design  will  be  reviewed  by  consultants  hired  specifically  for  that 
purpose. 

3.  How  was  validation  of  computer  resource  requirements,  including  software, 

conducted? 


A.  Resources  were  added,  as  required,  until  all  requirements  were 
implemented;  a review  of  resources  showed  none  could  be  eliminated. 

B.  Computer  resources  required  were  the  same  as  those  in  the  weapon 
system  being  replaced. 

C.  Validation  was  performed  by  a special  team  of  contractor  personnel. 

D.  Validation  was  performed  by  a special  team  within  the  Program 
Manager's  staff. 

How  was  risk  analysis  performed? 

A.  Since  only  standardized  hardware  will  be  used,  no  risk  analysis  was 
required. 
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B.  Risks  in  software  and  hardware  were  estimated  after  discussing  the 
system  with  the  Service  organization  that  developed  similar  systems. 

C.  Risk  analysis  was  performed  by  consultants  hired  for  this  purpose. 

5.  How  will  you  insure  planned  computer  resources  will  meet  stated  operational 

requirements? 

A.  Good  estimates  of  resources  needed  to  meet  requirements  were  used. 

B.  Computer  resources  have  been  overspecified;  resources  not  needed 
immediately  will  be  available  for  future  expansion. 

C.  Difficult  operational  requirements  have  been  programmed;  additional 
memory  and  time  exist  for  remaining  simpler  requirements. 

6.  How  will  future  changes  to  computer  hardware  and  software  requirements  be 

made? 

A.  The  system  is  designed  so  that  no  future  hardware  and  software  will  be 
required. 

B.  The  Service  Headauarters  will  submit  changes  in  requirements  to  the 
embedded  computer  support  facility. 

C.  The  computer  support  facility  will  be  the  focal  point  for  all  hardware 
and  software  changes;  requests  for  minor  changes  will  come  from  the 
field,  while  requests  for  major  changes  will  come  from  the  Service 
Headquarters. 


DSARC  11  - Life  Cycle  Management  Issues 


1.  Which  DSARC  I Life  Cycle  Management  questions  are  still  unanswered? 
When  will  the  answers  be  known? 

A.  There  are  many  unanswered  quest  ions;  they  wii!  be  answered  during 
full-scale  engineering  development. 

B.  The  Service  Support  Agency  has  not  been  designated;  all  other  questions 
have  been  answered. 

C.  All  questions  have  been  answered. 

2.  Has  a Computer  Resources  Life  Cycle  Plan  been  written?  By  whom? 

A.  It  has  not  been  written. 


B.  It  was  written  by  the  Project  Manager's  Staff. 

C.  It  was  written  by  consultants  (via  a separate  contract). 

D.  It  was  written  by  the  Service  support  agency. 


3.  What  steps  have  been  planned  for  the  software  "turnover"  from  the 

contractor  to  the  government? 

A.  None;  the  Service  support  agency  is  not  known. 

B.  Software  turnover  will  be  coordinated  during  full-scale  engineering 
development. 

C.  None;  the  contractor  that  developed  the  software  will  support  it. 

D.  Detailed  schedules  have  been  prepared;  the  Service  support  agency  will 
provide  assistance  during  critical  design  reviews. 

U.  What  are  the  milestones  of  the  Computer  Resource  Life  Cycle  Management 

Plan?  What  criteria  will  be  used  to  measure  their  attainment? 

A.  Percent  project  completion  and  percent  of  available  funds  spent. 

B.  Delivery  of  documents  from  the  contractor. 

C.  Delivery  of  documents  from  the  contractor  after  a thorough  review  of 
draft  copies. 

5.  How  will  the  computer  resources  be  integrated  into  the  total  defense  system? 

A.  This  problem  will  be  addressed  when  the  capabilities  and  limitations  of 
the  final  system  have  been  demonstrated. 

B.  Once  they  are  fixed,  input/output  data  formats  will  be  given  to  the 
other  services  so  their  systems  can  be  made  compatible. 

C.  (List  of  detailed  plans  to  include  coordination  within  the  Service  and 
with  other  Service  members,  training  for  operators  and  maintenance, 
personnel,  logistic  support,  etc.) 

6.  How  will  the  overall  system  quality  be  determined? 

A.  The  quality  of  the  system  will  evolve  as  the  system  develops. 

B.  The  system  will  be  measured  against  the  original  operational  require- 
ments in  a series  of  laboratory  simulations. 

C.  The  system  will  be  measured  against  the  original  operational  require- 
ments in  field  tests. 


\ 
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7.  How  were  personnel  requirements  lor  supporting  computer  resources  deter- 
mined? 

A.  The  number  of  people  needed  to  develop  the  system  will  be  needed  to 
support  it. 

B.  Fifty  percent  of  the  programmers  needed  for  development  will  be 
needed  for  support. 

C.  (A  reasonable  estimation  based  not  only  on  programmers,  but  also  on 
analysts,  hardware,  interfacing  and  test  personnel.) 

8.  What  software  is  contract  deliverable? 

A.  Operating  programs  in  object  language  are  the  only  deliverables. 

B.  Operating  programs  in  source  language. 

C.  Operating  programs  in  source  language  plus  the  compiler  and/or  the 
assembler  used,  development  flow  charts,  and  other  documentation. 

D.  The  items  listed  in  response  C plus  the  simulation  and  configuration 
management  software  used  during  development. 

DSARC  II  Tradeoff  Issues 


1.  How  were  tradeoff  decisions  made? 

A.  Since  no  changes  in  operational  requirements  are  expected,  hardware 
was  used  as  much  as  possible. 

B.  Software  was  used  instead  of  hardware  whenever  possible  so  changes 
can  be  made  easily. 

C.  Time-critical  subfunctions  were  implemented  in  hardware,  the  remain- 
ing subfunctions  in  software. 

D.  Life  cycle  costs  for  software  and  hardware  were  used  to  make 
hardware/software  decisions. 

2.  Did  the  user  team  that  wrote  the  original  operational  requirements  assist  in 

cost  versus  capabilities  tradeoff?  If  not,  how  were  these  tradeoffs 

evaluated? 

A.  No;  tradeoffs  were  done  by  one  of  the  Service  laboratories. 

B.  No;  cost /capabilities  tradeoffs  were  performed  by  personnel  from  the 
Program  Manager's  Staff. 
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C. 

No;  tradeoffs  were  analysed  by  a team  of  future  users,  support 
personnel,  Program  Manager  personnel,  and  prime  contractor  personnel. 

D. 

Yes 

DSARC  II  - 

Program  Manager's  Staff  Issues 

r — 

l. 

What  percentage  of  development  costs  will  be  spent  on  computer -related 
expenses? 

A. 

An  insignificant  amount. 

B. 

Five  percent. 

C. 

Twenty-five  percent 

2. 

How  many  dedicated  program  personnel  are  skilled  in  computers  and 
software?  What  percentage  of  the  staff  does  this  represent? 

A. 

None;  zero  percent. 

B. 

Four  persons;  six  percent. 

C. 

Twenty  persons;  thirteen  percent 

3. 

How  many  dedicated  program  personnel  have  had  operational  experience  in 
the  project  application  area? 

A. 

Only  a few. 

B. 

Less  than  twenty-five  percent. 

C.  Greater  than  seventy  percent. 

k.  What  plans  have  been  made  to  obtain  computer  personnel  temporarily  from 

Service  laboratories  and  support  activities?  From  private  consulting  firms? 

A.  No  such  plans;  no  computer  or  software  problems  are  invisioned. 

B.  The  Program  Manager's  staff  contains  sufficient  computer  personnel  to 
adequately  monitor  development  of  all  computer  hardware  and  soft- 
ware. 

C.  Contracts  have  been  awarded  to  private  consulting  firms  to  check  on 
software  costs  and  schedules  submitted  by  the  weapon  system 
contractor. 
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D.  Computer  personnel  from  the  Service  support  agency  will  review 
computer  system  design,  coding,  and  testing  to  insure  they  can  maintain 
the  software  when  the  system  is  deployed. 

E.  Several  computer  personnel  from  Service  laboratories  have  been 
assigned  to  the  Program  Manager's  staff  during  the  demonstration  and 
validation  phase  and  will  remain  assigned  during  most  of  the  full-scale 
engineering  development  phase. 

5.  Does  the  Project  Manager  have  an  experienced  system  engineer  agent 

responsible  for  overseeing  software  systems  engineering? 

A.  No;  most  software  was  obtained  from  a similar  system. 

B.  No;  all  software  design,  coding,  and  testing  will  be  handled  by  the 
contractor. 

C.  No;  overseeing  of  software  development  is  an  extra  task  performed  by 
the  chief  engineer  in  charge  of  the  weapon  system  hardware. 

D.  Yes;  this  individual  is  the  focal  point  for  all  software  development. 

6.  How  will  the  Program  Manager  provide  for  maintenance  support  require- 
ments? Is  there  a dedicated  Software  Operational  Support  Agency? 

A.  The  contractor  was  told  to  consider  maintenance  during  design  of  the 
system;  the  Service  software  support  activity  is  not  known  so  a 
dedicated  support  agent  is  not  required. 

B.  Personnel  from  the  Service  software  support  activity  attend  all  reviews 
to  be  sure  maintenance  support  is  adequately  considered;  no  dedicated 
support  agent  is  required. 

C.  Maintenance  support  requirements  were  determined  by  reviewing 
similar  systems;  the  requirements  were  then  incorporated  into  the 
contractor's  Statement  of  Work;  there  is  a dedicated  staff  person  acting 
as  software  operational  support  focal  point. 

* 

DSARC  II  - Project  Control  Issues 

1.  What  management  procedures  will  be  used  to  control  software  development? 

How  will  they  monitor  costing  and  scheduling? 

A.  The  actual  versus  projected  percentage  of  programming  completed  will 
be  compared  on  a monthly  basis  for  each  major  computer  subprogram; 
costs  will  also  be  compared. 

B.  Software  development  will  be  controlled  by  means  of  system  require- 
ment and  system  design  reviews  and  the  documentation  associated  with 
these  reviews. 
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C.  Software  development  will  be  controlled  by  comparing  actual  to 
estimated  completion  dates  and  actual  to  estimated  development  costs 
for  all  programming  modules  in  the  system. 

D.  PEilftyCPM  will  be  used,  with  the  development  of  each  major  software 
module  being  an  event  in  the  network. 

What  milestones  have  been  chosen  for  the  management  plan? 

A.  All  milestones  are  based  on  percentages  of  total  programming  complet- 
ed. 

B.  Milestones  consist  of  receiving  written  documents. 

C.  Milestones  consist  of  receiving  final  copies  of  selected  documents  that 
have  been  previously  reviewed  for  content,  completeness,  accuracy, 
etc. 

D.  Milestones  consist  of  easily  recognized  low  level  events  in  software 
development  such  as  freezing  of  design  specifications,  passing  of 
operational  tests,  etc. 

Will  there  be  any  parallel  software  development  efforts?  If  so,  how  will  they 

be  controlled? 

A.  A chief  programmer  will  control  all  teams  doing  parallel  development. 

B.  Weekly  meetings  of  programming  team  leaders  will  be  held  to  improve 
communications  among  teams  doing  parallel  development. 

C.  Programming  standards  will  state  which  parameters  will  be  passed  to 
modules  developed  by  different  teams;  storage  of  shared  data  will  also 
be  covered  in  standards. 

D.  Relatively  little  software  is  needed;  no  parallel  efforts  are  planned. 

How  will  interface  control  be  handled? 

A.  External  hardware  to  be  used  has  not  been  determined;  software  will  be 
modified  when  hardware  specificatons  are  available. 

B.  The  weapon  system  will  contain  many  external  devices;  interfacing  to 
each  device  will  be  handled  on  a case  by  case  basis. 

C.  An  Interface  Design  Specification  will  be  provided  by  the  contractor  at 
start  of  full-scale  engineering  development;  it  will  contain  complete 
specificatons  of  all  interfaces. 

D.  interfacing  information  for  all  Government  Furnished  Equipment  has 
been  provided  to  the  contractor;  the  contractor  will  handle  all  internal 
interfacing. 
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DSARC  II  - Development  Contract  Issues 


1.  Will  acquisition  take  place  in  accordance  with  Public  Law  89-306?  Why 

or  why  not? 

A.  It  will  depend  on  how  the  contractor  desires  to  acquire  the  computing 
system. 

B.  The  computer  will  not  be  acquired  in  accordance  with  PL  89-306 
because  it  is  not  an  "off-the-shelf"  system. 

C.  The  ground  support  computer  will  be  acquired  under  PL  89-306;  the 
airborne  computer  will  not;  determination  was  based  on  where  the 
computer  will  be  located. 

D.  The  computers  were  not  acquired  in  accordance  with  PL  89-306  because 
they  are  embedded  within  the  weapon  system. 

2.  Which  type  of  contract  will  be  employed  for  the  software? 

A.  Software  does  not  have  a separate  contract  but  is  part  of  the  hardware 
contract. 

B.  Fixed  cost  contract. 

C.  Cost  plus  fixed  fee  contract. 

D.  Cost  plus  fixed  fee  with  incentative  for  early  delivery. 

3.  How  will  the  contractor  be  tasked  for  software  items? 

A.  All  software  is  treated  as  a single  configuration  item. 

B.  Operational,  developmental,  and  system  support  software  are  each 
handled  as  separate  configuration  items. 

C.  The  contractor  will  develop  a list  of  software  items  which  will  be 
reviewed  by  the  Program  Manager  prior  to  tasking  for  software  items. 

D.  Separate  software  consultants  will  determine  the  software  tasks  that 
must  be  performed. 

9.  What  will  be  the  software-related  contractor  incentives? 

A.  There  will  be  none. 

B.  Incentives  will  be  given  for  early  completion. 

C.  A percentage  of  difference  of  estimated  and  actual  software  develop- 
ment cost  will  be  paid  to  contractor. 
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5.  How  will  "negative  software  incentives"  (minimizing  hardware  at  the  expense 

of  software)  be  handled? 

A.  After  development,  software  will  be  supported  by  the  contractor  on  a 
fixed  fee  basis. 

B.  Separate  consultants  will  determine  whether  software/hardware  trade- 
offs have  been  made  in  the  proper  manner. 

C.  The  contractor  incentive  fee  will  be  based  on  life  cycle  costs  that 
include  software  maintenance. 

D.  There  are  separate  contracts  and  incentivesdfcr  software  and  hardware. 

6.  Is  all  software  listed  as  configuration  item(s)?  Which  software  is  not 

deliverable? 

A.  No;  the  contractor  implemented  several  functions  with  microprocessors 
rather  than  with  hard-wired  logic;  this  microprocessor  software  is  not 
included  in  the  contract. 

B.  No;  development  software  was  previously  developed  by  the  contractor 
at  his  own  expense  and  will  not  be  delivered  with  the  operational 
software. 

C.  Yes. 

7.  Is  all  support  software  listed  as  deliverable?  Is  any  proprietary?  If  so,  how 

will  this  behandled? 

A.  Software  will  be  maintained  by  the  contractor  so  there  are  no 
proprietary  software  problems. 

B.  A proprietary  high-level  language  preprocessor  will  be  used  during 
software  development;  output  of  this  preprocessor  is  in  acceptable  form 
for  required  documentation. 

C.  Arrangements  have  been  made  with  the  contractor  to  lease  proprietary 
software  when  it  is  needed  for  software  support  and  maintenance. 

D.  All  software  is  listed  as  deliverable. 


DSARC  D - Testing  Issues 


1.  When  will  the  system  and  program  designs  be  frozen? 

A.  Not  known;  the  desired  capabilities  of  the  weapons  system  are 
continually  changing. 

B.  Near  the  end  of  full-scale  engineering  development. 
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C.  System  design  is  now  frozen,  program  design  will  be  frozen  early  in  full- 
scale  engineering  development. 

D.  System  and  program  designs  have  been  frozen  except  for  certain  key 
modules  which  are  expected  to  change  (due  to  changing  enemy  threats) 
during  full-scale  engineering  development. 

E.  All  system  and  program  designs  have  been  frozen. 

How  will  software  testing  be  performed? 

A.  Personnel  developing  the  coding  will  test  it  as  it  is  being  developed; 
each  programmer  will  generate  his  own  test  data. 

B.  Testing  will  be  performed  by  a different  group  than  the  one  that 
developed  the  programs;  the  testing  group  will  generate  the  test  data 
based  on  the  system  functional  requirements. 

C.  The  contractor  will  develop  a test  plan  for  testing  of  all  software;  this 
plan  will  be  reviewed  by  Program  Manager's  staff  for  adequacy. 

D.  Software  testing  plans  were  developed  by  the  contractor  during  the 
demonstration  qnd  validation  phase;  several  modules  that  will  be  used  in 
the  delivered^ystem  have  been  tested  in  accordance  with  this  plan. 

How  will  you  insure  the  test  data  is  representative  of  the  total  range  of  data 

ana  conditions  that  the  system  might  encounter? 

A.  Program  test  data  was  developed  for  all  "worst  cases". 

B.  Hot-beds  are  to  be  used  to  simulate  conditions  that  will  exist  in  the 
actual  environment,  including  noise. 

C.  Programs  were  monitored  to  check  that  all  code  was  used  during 
testing;  when  necessary,  special  data  was  created  to  tes*  all  code. 

D.  In  addition  to  laboratory  testing,  the  system  was  given  preliminary 
testing  in  the  field. 

Is  there  a software  module  test  plan  and  a software  module  test  procedure? 

A.  No;  since  there  is  relatively  little  software  in  the  system,  formal  test 
plans  and  procedures  are  not  required. 

ft  No;  the  contractor  policy  on  software  testing  was  utilized. 

C.  Yes;  plans  were  developed  by  the  contractor  early  in  the  demonstration 
and  evaluation  phase  and  were  approved  by  the  Program  Manager. 
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hardware  related?  How  will  the  determination  of  whether  other  errors  are 
caused  by  hardware  or  software  be  made? 

A.  Software  will  be  fully  tested  using  software  emulation;  any  errors 
discovered  later  must  be  due  to  hardware. 

B.  Software  is  being  developed  by  engineers;  their  backgrounds  will  enable 
them  to  differentiate  between  hardware  and  software  errors. 

C.  In-circuit  emulators  will  be  used  to  record  the  sequence  of  events 
leading  up  to  errors;  trace  back  will  determine  source  of  errors. 

D.  Most  errors  will  be  discovered  using  hot-bed  testing;  conditions  causing 
errors  will  be  duplicated  until  a determination  of  the  type  of  error  can 
be  dter  mined. 

6.  Are  "hot-beds"  required  to  adequately  test  software?  Will  they  become 
government  property  after  testing  is  complete?  If  not,  does  the  government 
have  equivalent  integration  and  testing  facilities  available? 

A.  No;  software  can  be  tested  more  cost  effectively  using  software 
emulation. 

B.  Yes;  hot-beds  will  not  become  government  property  since  they  will  not 
be  needed  after  full-scale  engineering  development. 

C.  Yes;  hot-beds  will  remain  at  the  contractor's  site  since  he  will  provide 
software  updating  and  maintenance. 

D.  Yes;  hot -beds  are  to  remain  contractor's  property;  the  government  has 
equivalent  facilities. 

E.  Yes;  hot-beds  are  listed  as  contract  deliverable  items. 

7.  How  will  modules  be  interfaced  with  one  another?  How  will  these  interfaces 
be  tested? 

A.  All  parameters  used  by  more  than  one  program  module  are  located  in 
dedicated  memory  locations  available  to  all  modules;  no  interfacing 
between  modules  is  required. 

B.  The  software  will  be  developed  "top  down"  so  interfacing  will  not  be  a 
problem. 

C.  A special  software  team  will  interface  and  test  program  modules 
developed  by  other  programmers. 
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D.  Interface  specifications  are  developed  before  programming  begins; 
interfaces  are  tested  using  especially  generated  test  data. 

8.  What  critical  questions  and  areas  of  r»sk  still  need  resolving  by  testing?  What 

are  the  test  plans  and  milestones  for  resolving  these  problems? 

A.  Only  typical  cases  have  been  used  for  testing;  there  may  not  be  enough 
processing  speed  to  perform  necessary  calculations  during  worse  cases; 
test  plans  have  been  prepared  to  determine  and  to  test  the  worse  case. 

B.  A new  computation  algorithm  for  guidance  has  been  developed  based  on 
the  design  plan  for  a new  transducer;  since  the  new  transducer  has  not 
been  delivered,  the  software  cannot  be  fully  tested;  test  plans  with 
milestones  have  been  prepared. 

C.  Some  major  modules  may  have  to  be  increased  in  size  to  allow  for 
detection  of  errors  in  input  signals;  the  increase  in  memory  may  require 
more  physical  space  for  the  computer;  test  plans  include  testing  of 
revised  modules,  with  milestones  for  determining  whether  or  not  more 
space  will  be  required. 

D.  All  testing  has  been  performed  under  controlled  conditions  in  the 
laboratory  so  performance  in  the  field  is  unknown;  test  plans  include 
scheduled  tests  in  the  operating  environment  with  milestones  based  on 
successful  completion  of  the  tests. 

9.  How  will  test  related  documentation  be  maintained  to  allow  repeatability  of 

tests? 

A.  Individual  programmers  develop  and  test  their  own  modules;  all  test- 
related  information  is  kept  by  them. 

B.  All  testing  is  done  by  a separate  section  which  maintains  all  testing 
documentation;  documentation  on  testing  will  not  be  given  to  the 
government  under  the  existing  contract. 

C.  All  data  concerning  tests  will  be  recorded  in  contractor's  laboratory 
notebooks;  these  notebooks  will  be  available  from  the  contractor  as 
required. 

D.  Test-related  documentation  is  listed  as  a contract  deliverable  item. 


DSARC  II  - Software  Reliability  and  Maintainability  Issues 


1.  Will  one  of  the  high  order  languages  in  DoD  Instruction  5000.31  be  used  for 
programming?  If  not,  why  not?  What  percentage  of  the  software  will 
ultimately  be  written  in  assembly  language? 

A.  All  programming  must  be  written  in  assembly  language  due  to  limited 
memory  and  processing  time  available. 
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B.  Many  assembly  language  routines  from  previously  developed  systems 
will  be  utilized;  the  remaining  coding  will  have  to  be  in  assembly 
language  also. 

C.  FORTRAN  will  be  used  for  all  non-time-critical  programming  modules; 
60  percent  of  the  software  will  be  in  assembly  language. 

D.  All  programming  will  be  in  JOVIAL,  one  of  the  approved  languages. 

2.  How  will  you  insure  the  software  architecture  will  be  modular? 

A.  The  development  contract  states  all  software  must  be  modular. 

B.  Each  module  is  checked  by  a review  team  to  insure  modularity  and  good 
programming  practices  have  been  followed. 

C.  Each  module  is  limited  to  50  lines  by  the  editing  program  that  stores 
programs  being  developed. 

3.  How  will  you  insure  that  "top-down"  software  development  methodology  and 

that  structured  programming  will  be  used? 

A.  All  programmers  are  told  to  use  these  methods. 

B.  Both  techniques  are  required  in  the  contractor's  programming  standards 
manuals. 

C.  Both  are  part  of  contractor's  normal  method  of  programming. 

D.  Consultants  working  for  the  Program  Manager  have  been  checking  this 
during  milestone  review  meetings. 

k.  What  programming  standards  and  conventions  will  be  used?  How  will  they  be 

enforced? 

A.  Each  programming  team  develops  its  own  standards  based  on  what  is 
most  efficient  for  the  team  members;  the  team  leader  is  in  charge  of 
enforcement. 

B.  The  contractor's  normal  programming  standards  and  conventions  will  be 
utilized;  enforcement  will  be  his  responsibility. 

C.  Programming  standards  and  conventions  were  stated  in  the  original 
Request  for  Proposals;  adherence  to  standards  will  be  checked  period- 
ically by  Program  Manager's  staff. 

D.  Completion  of  the  programming  standards  and  conventions  manual  was 
a milestone  in  the  computer  resource  life  cycle  management  plan;  all 
coding  will  be  reviewed  by  the  software  life  cycle  support  activity  to 
insure  standards  and  conventions  were  followed. 


1-25 


r 


E.  Standards  and  conventions  were  developed  by  independent  consultants 
working  with  the  contractor;  all  programs  are  checked  by  the  computer 
systems  that  store  the  programs  being  developed. 

5.  When  will  the  Data  Item  Index  be  prepared  and  how  will  it  be  updated?  How 

will  you  insure  the  documentation  will  be  adequate  for  life  cycle  mainten- 
ance? 

A.  Since  the  software  programs  are  small,  no  Data  Item  Index  is  required; 
likewise  adequate  documentation  will  not  be  a problem. 

B.  Since  all  programming  is  modular,  data  can  be  named  differently  in 
each  module  and  a Data  Item  Index  is  not  needed;  documentation  will  be 
reviewed  for  adequacy  by  the  software  life  cycle  support  agency. 

C.  The  Data  Item  Index  will  be  prepared  near  the  end  of  full-scale 
engineering  development  when  all  parameters  are  known;  documenta- 
tion will  be  checked  during  all  Program  Manager  reviews. 

D.  The  Data  Item  Index  has  been  prepared  and  is  being  updated  as 
parameters  are  added,  deleted,  and  modified;  Program  Manager 
consultants  will  review  documentation  during  full-scale  engineering 
development. 

6.  Which  automatic  debugging  tools  will  be  used  during  program  development? 

A.  None  are  required  because  the  program  is  very  small. 

B.  A debugging  package  will  be  provided  by  the  computer  vendor  to  run  on 
his  computer. 

C.  An  in-circuit  emulator  will  be  used. 

D.  Extensive  software  emulation  on  a large-scale  computer  will  be  used  for 
debugging. 

7.  How  will  error  data  be  collected  and  analyzed? 

A.  Since  the  programming  team  is  small,  informal  discussion  between 
programmers  will  be  used. 

B.  Program  Change  Request  forms  contain  a block  to  indicate  why  the 
change  was  needed;  this  information  is  used  by  the  lead  programmer  to 
determine  causes  of  errors. 

C.  Each  update  of  a program  under  configuration  management  must 
contain  the  reason  for  the  revision;  if  the  revision  is  due  to  an  error,  the 
type  of  error  must  be  given;  information  on  errors  is  automatically 
analyzed  by  the  computer. 
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D.  Error  data  is  collected  on  an  Error  Report  form  which  is  forwarded  to 
the  Service's  software  support  group;  the  support  group  analyzed  the 
data  and  prepares  Service-wide  analysis. 

8.  How  wiil  the  software  be  integrated  with  the  hardware  during  full-scale 

engineering  development? 

A.  After  the  software  is  tested  using  simulation,  it  will  be  put  on 
computing  hardware  that  is  being  developed. 

B.  Integration  of  hardware  and  software  will  be  done  using  "hot-beds"  that 
will  provide  real-time  signals  to  the  sensor  system  and  will  monitor  all 
output  signals. 

C.  All  software  development  is  being  done  on  computing  hardware  to  be 
used  in  the  weapon  system  (the  computer  being  used  in  an  off-the-shelf 
unit);  no  separate  software  integration  will  be  required. 

9.  How  will  software  be  documented  as  it  proceedes  from  concept  to  design  to 

the  final  operational  system? 

A.  Programmers  will  keep  their  own  informal  documentation  until  the  end 
of  full-scale  engineering  development,  when  formal  documentation  will 
be  prepared. 

B.  Software  documentation  will  consist  of  the  System  Specification,  the 
Development  Specification,  and  the  Product  Specification. 

C.  Documentation  will  include  the  System,  Development,  and  Product 
Specifications  and  the  Development  Test  and  Evaluation  Test  Plans  and 
Reports. 

, 10.  How  will  the  software  be  supported  in  the  field?  What  hardware  and  software 

will  be  needed  for  the  support  base?  How  will  it  be  procured? 

A.  Software  will  be  under  a one  year  contractor  warranty;  all  bugs  will  be 
out  of  the  system  by  then  so  software  support  will  no  longer  be  needed. 

B.  Software  will  be  supported  by  the  designated  software  support  activity; 
since  they  are  already  supporting  similar  systems,  they  should  have  the 
required  hardware  and  software  needed  to  support  another  system. 

C.  The  contractor  will  support  all  software  via  a separate  contract  after 
system  deployment;  the  contractor  already  has  all  required  support  base 
hardware  and  software. 

D.  Software  will  be  supported  by  the  previously  selected  agency;  hot- 
beds" will  be  procured  with  the  system,  but  proprietary  software  to 
support  the  system  will  have  to  be  leased  from  the  contractor. 
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11.  How  will  you  insure  accuracy  of  coding  to  available  listings? 


A.  Each  programmer  is  responsible  for  destroying  all  old  listings  when 
coding  is  changed. 

B.  New  listings  are  printed  from  the  source  coding  as  the  first  step  in 
making  any  program  change. 

C.  The  listings  for  each  program  are  kept  in  a special  notebook  which  is 
updated  after  every  change. 

D.  All  programs  will  be  in  a programming  library;  all  changes  must  be 
made  through  the  librarian  who  will  update  listings. 


DSARC  II  - Miscellaneous 


1.  What  has  the  contractor  done  of  a similar  nature  in  the  past?  What  were  his 

successes  and  failures?  What  is  he  doing  to  eliminate  past  problem  areas? 

A.  Most  of  the  contractor's  experience  has  been  in  the  hardware 
development  areas,  with  software  development  being  limited  to  much 
smaller  systems  than  the  current  one;  past  success  rate  has  been 
excellent;  he  has  hired  many  new  "software  engineers"  for  this  project. 

B.  The  contractor  has  developed  software  for  many  weapons  systems  of 
similar  complexity;  the  majority  of  past  systems  had  schedule  slippages 
and  cost  overruns,  mainly  due  to  changing  requirements;  the  contractor 
has  developed  a computerized  system  to  link  requirements  to  coding 
within  program  modules. 

C.  The  contractor  has  developed  many  systems  of  similar  complexity  but 
this  is  his  first  major  DoD  project;  past  projects  have  been  very 
successful;  management  methods  have  improved  during  the  past  several 
years. 

D.  The  contractor  has  developed  several  large-scale  systems  previously; 
the  major  problem  in  these  systems  was  the  support  of  delivered 
software  due  to  inadequate  program  documentation;  programming  and 
documentation  standards  have  been  and  will  continue  to  be  closely 
monitored. 

2.  What  problems  must  be  solved  prior  to  DSARC  III  that  have  not  already  been 

discussed?  What  is  your  plan  to  solve  them? 

A.  Finding  trained  software  engineering  personnel  for  the  Program 
Manager's  staff;  assistance  will  be  provided  by  Service  support  agencies 
and  nongovernment  consultants  until  personnel  with  the  right  qualifica- 
tions can  be  found  and/or  trained. 
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DSARC  III  QUESTIONS  AND  RESPONSES 


The  DSARC  III  decision  point  is  reached  when  the  production  recommendation 
for  the  system  can  be  made. 


DSARC  III  - General  Issues 


1.  Are  the  original  operational  requirements  still  valid?  How  can  this  be 

proven? 

A.  No;  operational  requirements  are  constantly  changing  due  to  changes  in 
enemy  capabilities. 

B.  No;  requirements  were  modified  during  full-scale  engineering  develop- 
ment as  a result  of  cost  versus  capabilities  analysis. 

C.  Yes;  the  service  has  not  changed  them  since  the  original  operational 
requirements  were  determined. 

D.  Yes;  the  Program  Manager  has  quarterly  meetings  with  the  heads  of 
organizations  that  will  be  using  the  system. 


DSARC  III  - Present  Status  Issues 


1.  What  are  the  results  of  the  latest  series  of  operational  tests  (on  the  entire 
weapon  system)?  Where  are  the  current  tests  in  relationship  to  the  overall 
test  plan? 

A.  Major  subelements  of  the  system  have  been  tested,  but  the  system  has 
not  been  tested  as  a whole  because  all  required  software  has  not  been 
produced.  The  current  test  plan  is  about  half  completed. 

B.  Testing  has  been  proceeding  well  except  for  a major  malfunction  that 
occurred  several  tests  ago.  The  reason  for  the  failure  could  not  be 
determined;  a duplicate  test  was  run  satisfactorily.  The  original  plan 
requiring  additonal  testing  was  changed  due  to  limitation  of  funds. 

C.  The  entire  system  has  worked  satisfactorily  during  the  latest  tests. 
Overall  tests  will  be  completed  after  the  first  few  systems  produced 
using  normal  factory  production  methods  are  tested  in  the  field. 


D.  Testing  has  indicated  that  the  system  is  ready  lor  production;  no  need 
lor  system  modilications  was  discovered  during  testing.  Testing  has 
been  completed. 

What  impact  will  the  need  lor  subsystem  changes  discovered  during  testing 
and  evaluation  ol  the  overall  weapon  system  have  on  embedded  computer 
hardware  and  soltware? 

A.  Operator  reaction  time  was  lower  than  anticipated;  the  embedded 
computer  soltware  will  have  to  be  modilied  so  the  amount  ol  operator 
interaction  is  reduced. 

B.  Additional  cooling  required  by  another  subsystem  will  reduce  the 
amount  of  cooling  available  lor  the  embedded  computer;  the  elfect  ol 
the  reduced  cooling  on  computer  reliability  is  not  known. 

C.  The  modifying  of  subsystems  during  testing  and  evaluation  created  the 
need  for  many  software  "patches";  these  patches  have  been  made  and 
tested. 

Are  any  software  modules  incomplete?  Which  modules  and  associated 
hardware  are  involved?  What  is  the  extent  of  incompleteness  and  the 
schedule  for  completion? 

A.  Approximatley  half  of  the  software  modules  have  been  written  and  fully 
tested.  Modules  dealing  with  calculating  parameters  for  the  guidance 
modules  from  sensor  inputs  have  not  been  completed;  these  modules  are 
scheduled  to  be  coded  and  tested  during  the  next  six  months. 

B.  All  modules  were  coded,  but  testing  indicated  a faster  processor  is 
required.  The  new  processor  selected  has  a different  instruction  set  so 
some  recoding  will  be  necessary;  recoding  and  testing  will  require  three 
months. 

C.  Several  guidance  modules  are  about  90  percent  completed;  they  should 
be  completed  within  the  next  several  months. 

D.  All  software  is  complete;  the  software  staff  has  been  reduced  to  a few 
programmers  to  correct  any  coding  found  to  contain  errors  during 
future  field  testing. 

What  is  the  profile  of  the  last  three  months  of  Discrepancy  forms  and 
Software  Change  Requests?  How  many  discrepancies  are  still  to  be 
corrected?  How  is  the  error  data  collected  and  analyzed? 

A.  The  number  of  software  changes  requested  has  remained  constant  at  a 
high  level  due  to  continual  changes  in  requirements.  Many  errors  had 
not  been  corrected.  Error  data  is  analyzed  by  individual  programmers 
who  made  the  errors. 


B.  The  number  of  discrepancies  discovered  and  software  changes  requested 
has  increased  recently  due  to  the  integration  of  many  software  modules 
into  the  total  system.  Most  discrepancies  have  been  corrected.  Error 
data  is  forwarded  to  the  Service  headquarters  for  analysis. 

C.  The  number  of  required  software  changes  needed  recently  .increased 
substantially  due  to  more  modules  being  put  under  configurator, 
management.  Most  discrepancies  have  been  corrected;  error  data  is 
analyzed  by  the  lead  programmer. 

D.  The  number  of  software  corrections  has  been  steadily  decreasing.  All 
known  errors  have  been  corrected.  Error  data  is  collected  and  analyzed 
by  a separate  branch  that  tests  program  modules. 

How  much  of  the  recent  software  change  activity  has  been  due  to  program 
errors  and  how  much  has  been  due  to  change  in  requirements?  Were  changes 
in  requirements  due  to  increased  or  decreased  requirement?  Who  has  the 
authority  to  change  software  requirements? 

A.  Approximately  60  percent  was  due  to  increased  requirements.  The 
Program  Manager  can  change  software  requirements. 

B.  Approximately  20  percent  was  due  to  decreased  requirements  (due  to 
insufficient  memory  to  fulfill  all  original  requirements).  The  Service 
headquarters  must  approve  all  software  requirement  changes. 

C.  All  recent  software  changes  have  been  due  to  program  errors.  Software 
requirements  have  been  frozen  and  can  only  be  changed  by  the  Service 
headquarters. 

How  has  delivered  code  been  verified  to  conform  to  original  software  design? 
Who  prepared  test  data  for  the  verification?  How  has  delivered  code  been 
shown  to  satisfy  original  operational  requirements? 

A.  The  original  software  design  was  modular  and  developed  in  a "top-down" 
fashion  and  thus  verification  was  automatic.  Test  data  for  verification 
was  prepared  by  a special  consulting  group  hired  by  the  Program 
Manager  for  that  purpose.  The  code  is  running  without  errors  so 
original  operational  requirements  must  have  been  met. 

B.  Individual  programmers  were  responsible  for  conforming  to  the  software 
design  and  for  preparing  their  own  test  data.  Delivered  code  was  used 
in  operational  tests  that  satisfied  original  operational  requirements. 

C.  A separate  testing  branch  prepared  all  test  data  and  verified  that 
delivered  code  conforms  to  the  original  test  design.  Computer  workload 
equivalent  to  the  worst  case  operational  requirements  was  used  t^  test 
the  software. 


D.  In  the  original  software  design  the  major  programs  for  each  operational 
requirement  were  identified;  during  full-scale  engineering  development 
each  major  program  was  broken  down  into  many  program  modules  which 
conformed  to  the  original  design.  Test  data  for  verification  was 
developed  by  the  system  designers  during  software  design. 

7.  How  was  hardware/software  integration  and  validation  performed? 

A.  Software  was  developed  on  large  scale  computers;  after  testing,  the 
entire  software  package  was  transferred  to  the  target  computer  and 
retested. 

B.  As  individual  modules  were  developed  and  tested  on  a large  scale 
computer,  they  were  transferred  to  the  actual  hardware  and  retested. 

C.  Since  very  little  software  was  required,  it  was  developed  on  the 
computer  hardware  that  will  be  used  in  the  weapons  system. 

8.  What  is  the  accuracy  of  coding  to  available  listings?  How  can  this  be 

demonstrated? 

A.  Listings  are  not  normally  kept.  New  listings  are  produced  by  the 
program  librarian  whenever  needed. 

B.  Very  good;  verified  by  checking  programmer’s  deck  with  his  listing. 

C.  Excellent;  this  can  be  verified  by  comparing  listing  kept  in  program 
development  notebooks  with  printed  listings  obtained  from  program 
librarian. 

D.  Excellent;  the  program  librarian  puts  a copy  of  each  updated  listing  in 
the  appropriate  program  workbook  whenever  a program  is  modified. 


PS  ARC  III  - Life  Cycle  Management  Issues 


L Are  any  life  cycle  management  questions  from  DSARC  II  still  unanswered? 
Why? 

A.  The  software  support  agency  has  not  been  selected  yet. 

B.  Personnel  requirements  for  supporting  the  software  have  not  been 
determined  because  the  amount  of  software  to  be  revised  annually  (due 
to  errors  and  changing  enemy  threats)  has  not  been  determined. 

C.  There  are  no  unanswered  questions. 

2.  Is  the  computer  resource  life  cycle  management  plan  on  schedule?  If  not, 
what  impact  will  this  have  on  the  entire  weapon  system  during  production  and 
deployment? 


A.  No;  the  software  has  not  been  fully  tested.  Systems  will  be  assembled 
with  Read  Only  Memory  hardware  that  contain  untested  programs. 
These  memory  elements  will  be  replaced  if  software  errors  are 
discovered  after  production. 

B.  No;  the  schedule  of  events  leading  to  government  software  support  has 
slipped  due  to  inadequate  software  documentation.  No  effect  on 
production  and  deployment  is  envisioned. 

C.  No;  the  computer  resource  life  cycle  management  plan  has  not  been 
fully  developed.  This  should  have  no  effect  on  production  and 
deployment. 

D.  The  plan  is  on  schedule. 

When  will  the  software  "turnover"  from  the  contractor  to  the  government 
take  place?  What  steps  have  to  take  place  before  the  turnover?  Is  the 
software  life  cycle  support  activity  prepared  for  the  turnover? 

A.  Once  software  is  completed,  no  revisions  are  planned.  No  software 
turnover  will  be  required. 

B.  ’ _r  will  take  place  after  the  two  year  warranty  period  expires. 

main  steps  in  the  turnover  must  include  transfer  of  all  source 
coding,  documentation,  support  software,  and  test  bed  equipment. 
When  the  time  comes  for  turnover,  the  activity  will  be  prepared. 

C.  Software  will  be  supported  in  the  field  by  the  contractor,  first  under 
warranty  and  then  by  a separate  contract. 

D.  Turnover  will  take  place  after  field  testing  of  the  first  production 
models  (assistance  in  removing  software  bugs  will  be  provided  by  the 
contractor  for  one  year  after  turnover).  The  main  items  to  be 
completed  before  turnover  include  hiring  of  additional  personnel, 
providing  required  software  training,  checking  final  program  document- 
ation, and  leasing  of  proprietory  support  software  from  the  contractor. 

Who  will  provide  software  support  during  deployment  of  the  system?  What 
items  will  be  required  in  the  support  base?  How  will  future  modifications  to 
baseline  software  be  handled? 

A.  The  contractor  will  correct  all  errors  discovered  during  the  one  year 
warranty  period.  No  other  software  support  should  be  required  after 
that  time. 

B.  The  contractor  will  provide  continuing  support  through  a separate 
support  contract.  He  already  has  the  needed  support  base.  Future 
modification  will  be  made  on  an  annual  basis. 

C.  The  software  life  cycle  support  agency  selected  prior  to  DSARC  II  will 
provide  long  term  support.  Items  that  must  be  included  in  the  support 
base  are  an  assembler  (or  cross-assembler),  a compiler,  an  editor,  a 
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loader,  and  debugging  programs.  Future  modificatons  will  be  made 
when  requested  by  the  users  of  the  weapon  systems. 

D.  Software  will  be  supported  by  the  Service  software  support  agency. 
Items  required  include  trained  computer  analysts  and  programmers, 
source  coding  and  documentation,  large-scale  computer  with  support 
software,  and  equipment  to  provide  appropriate  input  signals  and  record 
output  signals.  Requests  for  software  modification  will  be  analyzed 
quarterly  to  determine  when  changes  will  be  made. 

5.  What  will  be  the  impact  of  anticipated  software  improvements?  What  are  the 

anticipated  improvements  and  which  areas  of  the  system  will  be  involved? 

A.  No  software  improvements  are  anticipated;  however,  some  coding  may 
have  to  be  modified  in  the  data  conditioning  modules  if  sensors  are 
improved. 

B.  During  the  estimated  four  year  production  run,  software  will  be 
simplified  when  a more  powerful  microprocessor  replaces  the  one  now 
in  the  system;  the  simplified  software  will  require  less  memory. 

C.  Improved  error  detection  software  will  be  developed  and  added  to  the 
system  during  the  first  annual  software  revision;  changing  of  some  Read 
Only  Memories  will  be  required. 

D.  Modular  programming  techniques  were  utilized  to  make  improving 
software  straight  forward;  no  major  impact  due  to  software  improve- 
ment is  expected. 

6.  What  is  the  general  logic  flow  for  the  system?  How  would  government 

personnel  go  from  the  general  flow  chart  to  the  source  coding?  is  a Data 

Item  Index  a deliverable  item? 

A.  The  general  logic  flow  is  described  in  the  Type  II  Specifications.  Each 
block  in  the  general  flow  chart  represents  from  50  to  4,000  lines  of 
program  coding.  Associated  with  each  block  are  the  names  of  the 
subprograms  that  contain  the  source  coding.  A Data  Item  Index  was  not 
used  during  program  development,  because  one  was  not  required  in  the 
original  contract. 

B.  General  logic  flow  is  described  in  the  system  documentation.  Each 
element  in  the  general  flow  chart  is  a subroutine  which  contains  the 
source  coding.  A Data  Item  Index  is  a deliverable  item. 

C.  The  general  logic  flow  diagram  is  shown  in  the  first  section  of  the 
systems  specification  documentation.  Since  the  software  was  developed 
using  the  "top-down"  method,  the  desired  module  in  the  general  logic 
flow  diagram  leads  to  a lower  level  module  which  leads  to  a still  lower 
one  until  the  desired  source  coding  is  located.  A Data  Item  Index  has 
been  developed  and  will  be  delivered  to  the  software  support  agency. 
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7.  How  is  the  software  compatible  with  operation/Jogistics  concepts? 

A.  All  software  was  written  in  a Higher  Order  Language  to  make  software 
modification  less  expensive  over  the  life  cycle  of  the  weapons  system. 

B.  Software  will  be  stored  in  Read  Only  Memory  chips  which  will  be 
distributed  via  normal  logistics  channels. 


DSARC  III  * Miscellaneous 

L Linder  what  conditions  will  a formal  operational  test  and  evaluation  be  re- 
quired for  major  computer  hardware  and/or  software  changes  made  after 
deployment  of  the  weapon  system?  How  will  it  be  funded? 

A.  No  major  computer  hardware  or  software  changes  are  expected. 

B.  Several  years  after  deployment,  the  embedded  computer  unique  to  the 
weapon  system  will  be  replaced  by  a military  standard  computer  with  a 
standard  executive  software  package;  a formal  operational  test  and 
evaluation  will  be  required.  Funding  will  be  provided  by  the  office 
managing  the  conversion  to  the  military  standard  computer. 

C.  A formal  operational  test  and  evaluation  will  be  required  if  any  change 
requires  more  than  one-third  of  the  software  to  be  rewritten.  Funds  for 
testing  will  be  included  in  the  request  for  additional  funds  needed  to 
implement  the  change. 
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PART  II 

SENSITIVITY  DATA  AND  RELATIONSHIPS 
PRODUCTIVITY  INFLUENCE  FACTORS 


Customer  - Contractor  Interface  Complexity 
User  Participation  in  Requirements  Definition 
Program  Design  Changes 

Customer  Experience  with  Application  Area 
Contractor  Personnel  Experience  and  Qualification 
Programmer  Participation  In  Functional  Design 
Previous  Experience  with  Operational  Computer 
Previous  Experience  with  Programming  Language 
High  Order  Language  Versus  Assembly  Language 
Previous  Experience  with  Bigger  Programs 
Classified  Environment  Effect 

Staff  Size  (Efficiency  and  Communication  Paths) 

Structured  Programming  Effect 

Design/Code  Inspection  Effect 

Top  Down  Development  Effect 

Chief  Programmer  Team  Effect 

Effect  of  Developed  Code  Complexity 

Percentage  of  Deliverable  Code 

Storage  Constraint  Impact 

Timing  Constraint  Impact 

Real  Time  Impact 

Non-Mathematical  Code  Impact 

Data  Base  Dispersion 

High  Order  Languages  on  Small  Dedicated  Processors 
(large  production  volume)  - Break  Even  Cost  Criteria 


Figures  11-1  through  11-25  describe  the  parametric  relationship  between 
twenty-five  "influence  factors"  characteristic  of  a software  development 
effort  and  a relative  productivity  Index. 

As  used  In  the  figures,  productivity  is  measured  In  terms  of  Delivered 
Source  Lines  of  Code  per  Han-Month.  This  measure  thus  includes  the 
functional  activities  of  design,  development  and  testing,  In  addition  to 
coding.  In  using  the  productivity  Index,  It  should  be  assumed  that  the 
quality  of  the  final  product  (i.e.,  delivered  code)  Is  fixed  at  a median 
level  of  three  faults  per  1000  lines  of  source  code. 

Developments  which  attempt  to  stress  the  productivity  Index  (i.e.,  force 
higher  productivity  without  appropriate  shift  on  the  influence  factor 
abc i ssa) .wi 1 1 probably  experience  a quality  distortion  of  the  three  fault/ 
1000  line  median.  Should  this  occur.  Figure  H-26  Illustrates  the  cost 
impact  of  detecting  and  correcting  a fault  after  del i very. 

Of  course,  many  other  factors  prevading  the  software  design,  development, 
implementation  and  test  environment  affect  the  contractor  productivity, 
and  hence  the  probability  of  meeting  realistically  set  schedule,  cost, 
and  quality  requirements.  However  strong  ones  intuitive  feeling,  though, 
there  is  little  empirical  evidence  in  hand  to  date  that  would  suggest  these 
other  factors  to  have  more  than  a secondary  influence  on  productivity, 
schedule,  cost,  or  quality. 
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RELATIVE  ERROR  CORREcr/OV 

COSTS 
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PHASE  IN  WHICH  ERROR  IS  OETECTEO 


PROGRAM  INQUIRY  QUESTIONS 


I.  Background  Information  (contractual) 

1.  Which  of  the  Modern  Program  Practices  given  below  are  used  In 

your  software  development  project?  (Check  those  that  apply.) 

Program  Manager  Authority — both  technical  end  administra- 
tive responsibility  for  project. 

_____  Reviews — formal  milestone  reviews  with  customer  participa- 
tion at  the  end  of  each  phase. 

_____  Unit  Development  Fol ders— -capture  of  working  materials  for 
each  identified  item  to  facilitate  end  item  development, 
testing,  documentation. 

_____  Design  Discipline  and  Verl f icet Ion— top  down  design,  formal 
design  representation,  completion  of  design,  and  deliberate 
verification  of  design  prior  to  code. 

____  Program  Modularity — def ini t ions/restr let  ions  on  data  Inter- 
faces between  modules,  adherence  to  parent/child  relation- 
ships between  modules'. 

Naming  Conventions — structured  names  for  modules  and/or 

data  Items. 

_____  Structured  Forms — use  of  Dijkstra  forms  as  supportable  In 
your  programming  language. 

Code  Verification — deliberate  peer  reviews  of  code  for  each 

module. 

_____  Support  Libraries  and  Faci 1 1 ties— -use  of  automated  or  pro- 
ceduralized  design,  coding,  and  configuration  management 
aids. 

Phased  Testing — defined  and  formalized  unit,  functional, 

and  acceptance  testing. 

Configuration  Management /Change  Control — creation  and  con- 
trol of  baselines  (requirements,  design,  Implementation) 
and  procedures  for  problem  reporting  and  resolution. 

2.  For  each  practice  checked,  supply  the  date  of  adoption  of  the 

practice  and  the  associated  software  development  phase  and/or 

milestone  at  which  It  was  adopted. 
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3.  For  each  practice  checked,  provide  a brief  rationale  which  telle 
why  the  practice  was  adopted  for  your  aoftware  development  pro- 
ject. For  each  practice  checked,  indicate  the  degree  of  euccees 
expected  or  achieved  In  the  project  by  Ita  adoption. 

4.  List  the  computing  hardware  (Including  the  host  and  target  machlni 
used  by  your  project  and  their  relat lonahlp. 

5.  List  the  operating  system(s)  and  compilers/assemblers  and  utility 
software  used  by  your  project. 

6.  Indicate  the  method(s)  of  access  to  the  hardware  and  the  software. 
(Check  those  which  apply.) 

Batch 

____  Remote  job  entry 

Time  sharing 

Stand-alone 

7.  Describe  the  availability  of  hardware/software:  the  time  schedule 
for  computer  accessibility  and  the  existence  of  any  restricted 

or  experimental  software/hardware  used  by  the  project. 

8.  Indicate  the  number  and  type  of  personnel  for  your  project.  (Enter 
number  which  applies.) 

___  Full  time,  report  to  Program  Manager 

Full  time,  report  outside  Program  Manager's  organization 

Full  time,  outside  contract 

Pert  time,  report  to  Program  Manager 

Part  time,  report  outside  Program  Manager's  organization 

(e.g.,  consultants) 

___  Part  time,  outside  contractor 

9.  Provide  personnel  resumes  which  detail  experience  end  training, 
both  prior  to  and  also  during  this  project  development. 

10.  Indicate  the  percentage  of  personnel  turnover  expected/experi- 
enced on  your  project.  Was  this  turnover  planned  for?  Did  It  have 
a detrimental  effect  on  project  performance? 
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11.  Characterize  your  project  in  terms  of  magnitude,  complexity  end 
software  type  (e.g.,  real  time,  utility,  application)  performed 
by  the  resultant  software.  When  did  your  project  begin? 

12.  In  what  phase  of  the  software  development  is  your  project  current- 
ly? (Check  only  one. ) 

____  Requirements  Definition 

Design 

Coding 

___  Checkout  end  Unit  Testing 

Integration  and  Testing 

System  Testing  and  Delivery 

_____  In  Production  (Operation  and  Maintenance) 

II.  Software  Estimating  Guidelines 

1.  Estimate  how  much  of  the  total  deliverable  software  Is  of  each 
of  the  following  types: 

Mathematical  Operations  

Report  Generation  ___ 

Logic  Operations  ____ 

Signal  Processing/Data  Reduction  _____ 

Real  Time/Executive/Avionics  Interfaces  ____ 

How  many  Independent  programs  does  this  represent? 

2.  Estimate  the  total  number  of  deliverable  source  statements,  exclu- 
ding commentary.  (If  this  Is  an  enhancement  or  re  imp  I ementat ion, 
do  not  include  statements  which  do  not  have  to  be  recoded.) 

3.  Is  this  development  a re  imp  I ementat ion  of  an  existing  software 
design?  Is  this  development  a conversion  of  an  existing  software 
system? 

4.  Is  this  a follow-on  contract  with  your  current  customer  (e.g., 
major  enhancement)? 
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5.  How  many  deslgners/progremmers  support  this  development? 


More  than  20 
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6.  Is  a higher  order  language  being  used? 


not  at  all 


exclusively 


partial ly 


7.  Does  the  source  language  used  provide  a macro  capability? 

not  at  all  yes 

8.  Do  you  have  documentation  forms  to  aid  In  expression  of  designs, 
tests,  data  structures,  etc.? 

not  at  a I I yes 

9.  Does  the  computer  system  used  allow  for  on-line  programming  activi- 
ties? 


not  at  el 


yes,  code/date  entry 


yes,  debugging 

10.  Does  the  computer  system  provide  debugging  tools? 


dumps  on  I y 


other  (specify) 


11.  What  is  the  designer /programmer  experience  with  the  engineering 
or  technical  discipline  of  application?  (Insert  V.) 


Entry  level 


Moderate 


12.  Whet  is  the  actual  number  of  man-months  (or  man-hours)  expended 
(by  phase)  on  this  software  development  to  date?  What  Is  the  ac- 
tual dollar  cost  of  this  labor  expense  (by  phase)  to  date? 

13.  What  is  the  actual  amount  of  CRU's  expended  (by  phase)  by  this 
software  development  to  date?  What  is  the  actual  dollar  cost  of 
this  machine  expense  (by  phase)  to  date? 

14.  What  Is  the  milestone  schedule?  What  percent  of  the  totel  flow 
time  Is  scheduled  for  the  following  phases? 

Requirements  definition  . 


Design 
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Code 


Checkout  

Integration  and  functional  test  __ _ 

System  testing  and  acceptance  ____ 

15.  What  percent  of  the  total  {estimated)  manpower  required  Is  planned 
to  be  (has  been)  used  In  each  phase? 

Ml.  Indicators  of  Modern  Programming  Practice 

1.  What  Is  the  responsibility  of  the  Program  Manager? 

__ ___  Technical  (product  quality,  reliability) 

___  Make  task  assignments 

_____  Administrative  (budgetary) 

Evaluate  performance  of  personnel 

2.  Are  formal  task  assignments  provided  by  the  Program  Manager  to 
project  personnel? 

3.  Are  formal  phase  reviews  scheduled  and  conducted? 

Pertinent  items  Identified  and  available  at  each  review 

Review  objectives  specified  end  understood  by  participants 

___  Results  of  review  followed  up 

•/  ___  Project  and  customer  representatives  participate  In  each 

review 


The  objectives  end  procedures  for  this  activity  were  stated 

in  advance 

This  activity  was  performed  according  to  these  objectives 

and  procedures 

4.  Is  project  documentation  pertinent? 

Document  schedule  exists  for  review  end  completion 

Purpose  of  each  document  stated  and  Justified 


Documents  are  scheduled  so  that  the  project  can  uae  each 
completed  document  as  source  material  for  next  phase's  actlvl- 
ties 


Each  document  Is  reviewed  by  Its  Intended  audience 

__  Appropriate  user  content  and  language  are  determined  for 
and  contained  In  each  document 

Project  control  of  documentation  so  that  It  "tracks"  with 

requirements/design/ implementat ion  changes 

_____  The  objectives  and  procedures  for  this  activity  were  stated 
in  advance 

_____  This  activity  was  performed  according  to  these  objectives 
and  procedures 

5.  Are  Unit  Development  Folders  (UDF)  used  by  the  project? 

"Working  papers"  for  each  Item  are  captured  as  they  are 

created 

___  Project  procedures  establish  form  and  content  of  the  UDF's 

Their  contents  are  utilized  for  developing  other  Items,  com- 
pleting other  tests 

Project  controls  the  eccess/use  of  UDF's 


___  The  objectives  and  procedures  for  this  activity  were  stated 
in  advance 

This  activity  was  performed  according  to  these  objectives 

and  procedures 

6.  Does  the  project  employ  design  discipline? 

_____  Top  down  design  approach  specified 

Formal  design  analysis  and  representation  techniques  used: 

Static  design  representation  (design  trees) 

___  Dynamic  design  representation  (transition  diagrams) 
Other  (explain) 

_____  Design  refinement  methods  are  used  to  adapt  the  abstract 
design  model  to  the  computing  environment 
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Design  comp  I stsnsss  criteria  are  developed  and  specified. 
What  are  they? 


___  The  objectives  and  procedures  for  this  activity  were  stated 
In  advance 

This  activity  was  performed  according  to  these  objectives 

and  procedures 

7.  is  the  design  verified? 

_____  Peer  reviews  (structured  walkthroughs)  of  design  are  con- 
ducted 

The  objectives  and  method  of  conducting  peer  reviews  are 

stated  In  advance 

Design  reachability  and  connectivity  analyses  procedurized 

and  used 

Design  modularity  analyses  procedurized  and  used 

Mechanical  evaluation  method  used 

Project  control  (compliance  methods  and  verification)  over 

the  process  of  design  verification  exists 

The  objectives  and  procedures  for  this  activity  were  stated 

in  advance 

This  activity  was  performed  according  to  these  objectives 

and  procedures 

6.  Is  the  completed  design  utilized  to  discipline  the  code  construe- 

tion  process? 

Standard  definition  for  design  and  program  documentation 

is  establ ished 

Code  construction  plan  is  prepared  and  used 

___  Formal  design  review  Involving  customer  Is  held  prior  to 
star4  of  coding 

The  objectives  end  procedures  for  this  activity  were  stated 

in  advance 

This  activity  was  performed  according  to  these  objectives 

and  procedures 
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9.  Is  program  modularity  practiced? 

___  Content  and  format  of  program  specifications  are  established 

Modularity  criteria  specified  and  adhered  to 

_____  The  objectives  end  procedures  for  this  activity  were  stated 
In  advance 

This  activity  was  performed  according  to  these  objectives 

and  procedures 

10.  Are  structured  forms  used  in  the  code? 

Block  structures  used 

Permissible  logic  statements  defined,  used,  and  controlled 

Project  compliance  and  verification  procedures  control  the 

use  of  structured  forms 

The  objectives  and  procedures  for  this  activity  were  stated 

in  advance 

This  activity  was  performed  according  tc  these  objectives 

and  procedures 

11.  Are  formal  coding  conventions  being  followed? 

__  Syntactical  forms  defined  that  ere  (dls)el  lowed  for  each 
programming  language 

__  Procedures  for  accessing  external  data  and  handling  error 
cond i t i ons 

____  Naming  conventions  for  units  of  code  and  data  variables 

Project  control  procedures  for  code  organization  and  comments 

_____  The  objectives  and  procedures  for  this  activity  were  stated 
in  advance 

____  This  activity  was  performed  according  to  these  objectives 
and  procedures 

12.  Is  the  code  verifiable? 

The  required  testing  and  examination  for  each  unit  of  code 

are  documented 

Peer  reviews  of  the  code  are  procedurlzed  and  conducted 
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____  The  objectives  end  procedures  for  this  activity  were  stated 
in  advance 

___  This  activity  was  performed  according  to  these  objectives 
and  procedures 

13.  Are  design  aids  to  support  the  software  development  used? 

____  Formal  design  language  used  In  completing  the  design 
Manual  aids  used  (specify) 

___  Automated  aids  used  (specify) 

The  objectives  and  procedures  for  this  activity  were  stated 

in  advance 

This  activity  was  performed  according  to  these  objectives 

and  procedures 

14.  Are  code  construction  aids  to  support  the  software  development 
used? 

___  Structured  programming  practices  used  to  complete  the  c<-- 
___  Manual  aids  used  (specify) 

___  Automated  aids  used  (e.g.,  precompiler)  (specify) 

The  objectives  and  procedures  for  this  activity  were  stated 

in  advance 

This  activity  was  performed  according  to  these  objectives 

and  procedures 

15.  Are  configuration  management  aids  to  support  the  software  develop- 
ment used? 

Baselined  end  Items  are  Identified  and  controlled.  Indicate 

at  what  milestone/phase  baseline  control  was  established. 

Manual  aids  used  (specify) 

___  Automated  aids  used  (specify) 

The  objectives  and  procedures  for  this  activity  were  stated 

in  advance 

This  activity  was  performed  according  to  these  objectives 

and  procedures 
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16.  Does  the  project  perform  uni \l Integra? Ion  software  testing? 

_____  Uni U Integra? Ion  testing  Is  defined  In  a formal  test  plan 

There  ere  expected  results  and  pass/fall  criteria  defined 

___  The  software  correctly  Implements  the  design  when  this  test- 
ing Is  complete 

There  Is  a prob I em/error  reporting  system 

The  project  uses  compliance  and  verification  methods  to 

control  this  testing  phase 

_____  The  objectives  and  procedures  for  this  activity  were  stated 
in  advance 

____  This  activity  was  performed  according  to  these  objectives 
and  procedures 

17.  Does  the  project  perform  functional  software  testing? 

_____  An  Internal  review  of  Items  Is  held  to  judge  their  quali- 
ty and  completeness  prior  to  functional  testing 

A handover  of  Items  into  a controlled  configuration  Is 

made  formally  by  the  developers 

_____  An  Independent  agency  performs  functional  testing 

_____  The  project  produces  and  executes  formal  test  plans  and 
procedures  for  functional  testing 

There  are  expected  results  and  pass/fail  criteria  defined 

A realistic  dress  rehearsal  of  the  acceptance  test  is  per- 
formed as  the  final  functional  test 

_____  The  software  correctly  satisfies  the  requirements  when  this 
testing  Is  complete 

There  is  a problem/error  reporting  system 

_____  The  project  uses  compliance  and  verification  methods  to 
control  this  testing  phase 

___  The  objectives  end  procedures  for  this  activity  were  stated 
in  advance 

_____  This  activity  was  performed  according  to  these  objectives 
and  procedures 
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IB.  Docs  the  project  perform  acceptance  testing? 

. Acceptance  test  requirements  ere  part  of  the  project's  re- 
quirements baseline 

_____  Formal  acceptance  test  plan  and  procedures  are  developed 
and  user  concurrence  Is  obtained 

There  ere  procedures  which  allow  review  of  the  quality 

and  completeness  of  the  deliverables 

Project  and  customer  are  asked  If  ready  to  begin  acceptance 

test Ing 

_____  There  Is  a formal  deliverables  baseline  and  a committed 
schedule  for  acceptance  testing  as  a result  of  a formal 
rev i ew 

There  Is  an  acceptance  test  report  attesting  to  the  satis- 
factory conclusion  or  conditional  acceptance 

The  report  Is  signed  by  the  customer 

___  The  objectives  and  procedures  for  this  activity  were  stated 
in  advance 

___  This  activity  was  performed  according  to  these  objectives 
and  procedures 

19.  Does  the  project  use  controlled  end  item  baselines? 

Project  end  Items  by  phase  are  Identifiable  based  on  written 

procedure  I mechanisms 

____  Controlled,  review-established  requirements,  design  and 
implementation  baselines  exist  with  written  procedures 

There  are  mechanisms  for  updating  and  distributing  established 

base  I ines 

The  objectives  and  procedures  for  this  activity  were  stated 

In  advance 

This  activity  was  performed  according  to  these  objectives 

and  procedures 

2D.  Does  the  project  utilize  a problem  reporting  system? 

There  are  formal  procedures  for  reporting  problems,  errors, 

desired  improvements 
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____  There  ere  formal  procedures  for  identifying,  processing, 
and  tracking  problem  reports 

_____  There  ere  format  procedures  for  obtaining  or  distributing 
problem  report  status  Information 

_ A file  of  completed  or  In  progress  problem  reports  exists 

The  objectives  end  procedures  for  this  activity  were  stated 

in  advance 

This  activity  was  performed  according  to  these  objectives 

end  procedures 

21.  Are  baseline  change  control  boards  used  by  the  project? 

____  The  board  represents  project  management  end  customers 

_____  They  have  discretionary  end  budgetary  authority  to  control 
changes  to  the  baseline 

____  They  assess  proposed  changes,  end  resolve  reported  problems 
by  assigning  action  Items 

They  prioritize  and  control  the  changes  to  be  implemented 

They  authorize  baseline  updates  and  distribution 

The  objectives  and  procedures  for  this  activity  were  stated 

in  advance 

This  activity  was  performed  according  to  these  objectives 

end  procedures 
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SOFTWARE  ESTIMATING  GUIDELINES 


Step  1 — Determining  the  Software  Type 


The  estimating  task  should  begin  with  analysis  of  the  problem  to  be  solved. 
Generally,  the  type  of  software  to  be  developed  will  be  some  combination 
of  the  fol I owl ng : 

— Mathematical  Operations 
— Report  Generation 
— Logic  Opera* ions 

— Signal  Processing,  or  Data  Reduction 
— Real  Time,  or  Executive  (also.  Avionics  Interfacing) 


In  this  step,  one  should  estimate  how  much  of  the  total  software  will  be 
of  each  type,  and  then  apply  a reasonableness  test  to  whether  this  distri- 
bution Is  valid.  For  example,  It  would  be  unusual  for  more  than  10-15*  of 
the  software  to  be  in  the  real  time  category,  or  more  than  half  to  be  in 
the  report  generation  category. 

Step  2 — Estimating  the  End  Product  Size 

Next,  one  should  estimate  for  each  of  the  categories  determined  In  Step  1, 
how  much  new  code  must  be  written.  Unless  the  project  has  unusually  severe 
requirements  for  conmentary  within  the  code,  this  set  of  estimates  can  be 
simply  in  terms  of  executable  statements. 

While  not  executable  in  the  usual  sense,  statements  which  define  storage 
areas  (Fortran  COMMON,  for  example)  should  be  included  in  these  estimates. 
Ordinarily,  these  statements  will  be  a relatively  small  portion  (10*  or 
less)  of  the  total.  Care  should  be  taken,  however,  to  recognize  situations 
In  which  storage  definition  is  a significant  portion  of  the  total  task  — 
for  example,  when  the  amount  of  storage  available  is  quite  limited  and  spe- 
cial techniques  must  be  used  to  fit  the  data  into  the  space  allocated.  On 
the  other  hand,  it  should  be  recognized  that  storage  defining  is  usually 
done  only  once,  and  then  replicated  in  all  of  the  routines  that  use  the 
Storage. 

The  estimated  number  of  statements  to  be  coded  should  include  only  de I i ver- 
ab I e code;  that  number  may  be  significantly  smaller  than  the  total  number 
of  statements  produced,  due  to  the  necessity  of  creating  various  deve I opment 
aids  (test  drivers,  test  data  bases,  translators,  simulators,  ate.)  to  support 
a project.  The  adjustment  factors  of  Table3B-l  take  Into  account  the  creation 
of  such  aids.  (If  a development  aid  is  to  be  delivered  to  the  customer.  It 
must  be  considered  as  a deliverable  item  in  the  estimating  process,  because 
of  the  need  for  testing  and  documentation  of  the  tool  itself). 

The  results  of  this  step  should  be  compared  with  the  distribution  prepared 
in  Step  1,  analyzed  for  reasonableness,  and  adjusted  If  necessary  before 
proceeding  to  Step  3. 


The  preceding  steps  will  have  produced  a breakout  of  the  development  task 
in  terms  of  new  statements  that  must  be  coded  in  each  of  several  categories 
or  software  type,  in  this  step,  the  following  multipliers  can  be  applied  to 
estimate  the  amount  of  labor  required  to  produce  this  code. 

— Mathematical  55  6 man-months/1000  statements 

— Report  e*  8 man-months/1000  statements 

— Logic  w 12  man-months/1000  statements 

— Signal  » 20  man-months/ 1000  statements 

— Real  Time  «5  40  man-months/1000  statements 

These  factors  assume  that  a "statement"  Is  one  fully  checked  out,  tested, 
and  documented  statement  coded  In  the  selected  language.  The  choice  of  lan- 
guage can  have  a significant  effect  on  the  development  cost,  but  ordinarily 
affects  only  portions  of  the  total  task. 

Step  4 — Estimating  and  Expenditure  Distribution 


Typically,  the  final  costs  of  a software  development  activity  will  tend  to 
be  distributed  about  as  follows: 


— Requirements  Definition  5 

— Design  end  Specification  25 

— Code  Preparation  10 

— Code  Checkout  25 

— Integration  and  Test  25 

— System  Test  10 

The  distribution  shown  is  a "raw  di str /but  ion";  that  Is,  It  does  not  take 
into  account  such  factors  as  re- Imp  I ementat ion,  existence  of  sophisticated 
debug  tools,  etc.  These  factors  are  accounted  for  in  Table  6-1,  and  are 
applied  in  Step  5.  Note  that  the  distribution  to  Requirements  Definition 
and  Design  and  Specification  tasks  includes  documentation;  I.e.  the  user 
documentation  and  detailed  design  specifications  are  the  product  of  these 
two  tasks. 

Further,  while  the  percentages  shown  here  will.  In  general,  be  Indicative 
of  the  manpower  allocated  on  a project,  they  will  not  necessarily  represent 
the  actual  flow  time  or  calendar  time  scheduled  for  each  task.  The  adjusted 
estimates  of  Step  5 wl I I more  closely  approach  a flow  time  distribution 
end  could,  therefore,  be  used  as  a basis  for  schedule  preparation.  Documen- 
tation Is  a very  apt  example  of  this  problem;  while  the  actual  preparation 
of  the  technical  content  of  a document  is  correctly  represented  In  the  per- 
centages above,  typing  support  and  the  flow  time  end  effort  Involved  In 
producing  a finished,  printed  manual  Is  not  and  should  be  added  to  the  es- 
timates produced  by  these  guidelines. 
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In  this  step,  the  labor  estimates  developed  In  Step  3 should  be  broken  down 
by  task  — for  each  of  the  software  types  Involved  In  the  problem.  The  re- 
sulting £ by  6 matrix  (where  fl.  Is  the  number  of  different  software  types  end 
6 is  the  number  of  activities  from  Step  4)  of  individual  task  estimates  will 
be  further  adjusted  in  the  next  step  for  the  particulars  of  the  project. 

Step  5 — Adjusting  the  Labor  Estimates 

Tabletif-l  shows  multipliers  that  should  be  applied  to  Individual  estimates 
to  account  for  various  task  characteristics.  In  using  this  table,  It  should 
be  noted  that  the  multipliers  ere  task-specific;  for  example,  use  of  a higher- 
order  language  does  not  effect  Requirements  Definition  or  System  Test  tasks. 
Other  multipliers  are  cumulative;  for  example,  use  of  both  a higher-order 
language  end  macro  capability  results  In  a coding  cost  which  Is  only  27% 
of  the  cost  using  assembly  language  without  macros. 

On  the  subject  of  macros,  care  should  be  used  to  avoid  Including  In  the  es- 
timates the  effort  required  to  develop  any  special  algorithms  (for  instance, 
to  satisfy  unusual  sizing  or  timing  requirements).  Such  efforts  should  be 
estimated  separately  and  their  costs  added  to  the  basic  project  costs  estima- 
ted using  these  guidelines.  Use  of  the  algorithms  can  be  considered  equiva- 
lent to  using  macros,  however. 

Step  6 — Estimating  Computer  Time 

After  the  individual  task  estimates  have  been  adjusted  per  Step  5,  the  re- 
vised estimates  can  be  sunmed  to  arrive  at  a total  labor  cost  for  the  pro- 
ject. The  final  step  is  to  estimate  the  machine  time  that  will  be  used  during 
the  development  activity. 

The  most  widely  accepted  rule  of  th'mb  is  that  approximately  three  hours 
Of  stand-alone  computer  time  will  be  spent  per  man-month. 
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PART  V 
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April  26,  1976 
NUMBER  5000.29 


ASD(IfcL) 

Department  of  Defense  Directive 


SUBJECT  Management  of  Computer  Resources 

in  Major  Defense  Systems 

References:  (a)  through  (in)  are  listed  in  enclosure  4 

I.  PURPOSE 

This  Directive  establishes  policy  for  the  management  and 
control  of  computer  resources  during  the  development, 
acquisition,  deployment  and  support  of  major  Defense  6y6tems. 

II.  APPLICABILITY  AND  S00PE  - 

A.  The  provisions  of  this  Directive  apply  to  the  Office  of 
the  Secretary  of  Defense,  the  Military  Departments,  the 
Organization  of  the  Joint  Chiefs  of  Staff,  and  the  Defense 
Agencies  (hereinafter  referred  to  collectively  as  "DoD 
Components") . 

B.  Its  provisions  encompass  major  programs  of  Defense 
systems  acquisition,  as  designated  by  the  Secretary  of 
Defense  (described  in  section  II.  of  DoD  Directive 
5000.1,  reference  (a)).  In  addition,  it  provides 
principles  to  be  applied  in  the  acquisition  of  Defense 
systems  that  do  not  fall  in  the  "major  acquisition 
category." 

C.  Excluded  from  the  provisions  of  this  Directive  are 
general  purpose,  commercially  available  automatic  data 
processing  assets  as  defined  and  administered  under 
CMB  Circular  A-71>  DoD  Directives  4105.55,  4l60.19, 
and  5100.40  (references  (b),  (c),  (d),  and  (e)).  How- 
ever, when  feasible,  the  terms,  tools,  and  techniques 
employed  in  the  general  purpose  area  will  be  adopted 
or  adapted  to  support  management  of  computer  resources 
in  major  Defense  systems. 

III.  DURATION 


It  is  intended  that  the  policies  and  principles  embodied 
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Id  this  Directive  ultimately  be  assimilated  aa  as  Integral  part 
of  the  established  prooess  of  acqulrlog  major  Defense  eye teas. 

Therefore,  the  continuing  Deed  for  this  Directive,  and  all  organisa- 
tional Institutions  created  herein  shall  he  reviewed  blannually  with 
a view  tenwd  cancellation  after  6 years.  DoD  Directives  9000.1, 

5000.2,  and  5000.3  (references  (a),  (g),  and  (h))  will  he  modified  as 
appropriate,  to  reflect  this  assimilation. 

XV.  IKKIPITIOKS 

Kras  used  la  this  Directive  are  defined  in  enclosure  1. 

V.  POLICY 

A.  General 

1.  Annual  expenditures  by  DoD  an  the  design,  development,  acqui- 
sition, management,  and  operational  support  of  computer  resources 
embedded  within,  and  integral  to  weapons,  ccmsnunl  cat  Ions,  cem- 
nand  and  control,  and  Intelligence  sensor  systems  are  measurwd 
In  the  billions  of  dollars.  Unreliability,  particularly  of 
software,  dlmlnshes  DoD  mission  effectiveness  In  many  major 
Defense  systems. 

2.  Computer  resources  In  Defense  systems  must  be  managed  as  ale- 
ments  or  subsystems  of  major  Importance  during  conceptual, 
validation,  full-scale  development,  production,  deployment, 
and  support  phases  of  the  life  cycle,  with  particular  emphasis 
on  computer  software  sad  Its  Integration  with  the  surrounding 
hardware. 

B.  Requirements  Validation  and  Risk  Analysis 

Validation  of  computer  resource  requirements.  Including  soft- 
ware, risk  analyses,  planning,  preliminary  design,  security 
where  applicable  (DoD  Directive  5200.28,  reference  (f ) ) and 
Interface  control  and  Integration  methodology  definition  will 
be  conducted  during  the  Concept  Formulation  and  Program 
Validation  phases  of  Defense  system  development,  prior  to 
Defense  Systems  Acquisition  Review  Oouncll  (D6ARC;  H. 

lhls  analysis  must  assure  conformance  of  planned  computer 
resources  with  stated  operational  requirements. 

Risk  analysis,  preliminary  design,  hardvare /software  inte- 
gration methodology,  external  Interface  control,  security 
features  (DoD  Directive  5200.26,  reference  (f)),  and /life 
cycle  system  planning  shall  be  Included  In  the  review. 

Correctness  of  software,  reliability,  Integrity,  maintain- 
ability, ease  of  modification,  and  transferability  will  he 
major  considerations  in  the  Initial  design. 

/ 

The  risk  areas,  and  a plan  for  their  resolution  shall  he 
Included  In  the  Decision  Coordinating  Paper  (DoD  Directive 
5000.2,  referenoe  (g)). 

In  addition,  computer  resouroe  requirements' will  he  continu- 
ously coordinated  and  reconciled  with  system  operational 
requirements  throughwt  system  development  after  DEARC  H. 

C.  Configuration  Management  of  Computer  Resources.  Defense  system 
computer  resources,  Including  both  computer  hardware  and  computer 
software  will  he  specified  and  treated  as  conf Iguratlon  items. 
Baseline  Implementation  guidance  for  this  action  Is  contained  In 
DoD  Instruction  5010.21  (reference  ( 1 ) ) . 

/ 

D.  Computer  Resource  Life  Or c le  Planning.  A computer  resouroe  plan 
will  he  developed  prior  to  DBARC  H,  and  will  he  maintained 
throughout  the  life  cycle.  The  purpose  of  the  plan  la  to  Identify 
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important  Defense  system  computer  resources  acquisition  and  life 
cycle  planning  factors,  both  direct  and  indirect;  and  to  establish 
specific  guidelines  to  ensure  that  these  factors  are  adequately 
considered  in  the  acquisition  planning  process.  Examples  of  factors 
to  be  addressed  are  the  following,  a6  applicable: 

1.  Responsibilities  for  integration  of  computer  resources  into  the 
total  Defense  system  and  the  determination  of  overall  system 
quality  and  integrity. 

2.  Personnel  requirements  for  developing  and  supporting  computer 
resources . 

3.  Computer  programs  required  to  support  the  development,  acquisi- 
tion, and  maintenance  of  computer  equipment  and  other  computer 
programs . 

A.  Provisions  for  the  transfer  of  program  management  responsibility 
after  initial  system  operating  capability  has  been  achieved;  pro- 
visions for  system/equipment  turnover. 

Support  Software  Deliverables.  Unique  support  items  required  to  cost 
effectively  develop  and  maintain  the  delivered  computer  resources 
over  the  system’s  life  cycle  will  be  specified  as  deliverable,  with 
DoD  acquiring  rights  to  their  design  and/or  use.  Examples  of  such 
support  items  are  compilers,  environmental  simulators,  documentation 
aids,  test  case  generators  and  analyzers,  and  training  aids.  The 
provisions  of  ASPR,  section  IX  (reference  (j))  shall  govern  the 
implementation  of  the  policy. 

Milestone  Definition  and  Attainment  Criteria.  Specific  milestones 
to  manage  the  life  cycle  development  of  computer  resources,  including 
computer  system  and  support  software  will  be  used  to  ensure  the  proper 
sequence  of  analysis,  design,  implementation,  integration,  test,  docu- 
mentation, operation,  maintenance,  and  modification.  These  milestones 
will  include  specific  criteria  that  measure  their  attainment. 

Sofware  Language  Standardizat ion  and  Control.  DoD  approved  High  Order 
Programming  Languages  (HOLs),  (reference  (k))  will  be  used  to  develop 
Defense  system  software,  unless  it  is  demonstrated  that  none  of  the 
approved  HOLs  are  cost  effective  or  technically  practical  over  the 
system  life  cycle.  Each  DoD  approved  HOL  will  be  assigned  to  a 
designated  control  agent  who  will  be  responsible  for  such  activities 
as  validating  compliance  of  compiler  implementations  with  the  standard 
language  specifications,  gathering  data  as  to  the  use  of  the  language, 
and  for  disseminating  information,  compilers,  and  tools.  The  designated 
control  agent  will  also  be  responsible  for  assuring  language  stability 
except  for  DoD  HOL  specifications  which  already  fall  within  the  purview 
of  DoD  Manual  A120.3M  (reference  (m)). 
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VI. 


RESPONSIBILITIES 


A.  In  order  to  oversee  and  coordinate  the  accomplishment  of  poli- 
cies in  this  Directive  and  the  incorporation  of  its  principles 
into  the  normal  Defense  system  acquisition  process,  a Manage- 
ment Steering  Committee  for  Embedded  Computer  Resources  is 
hereby  established.  This  Conmittee  shall  operate  under  the 
Charter  of  enclosure  2 for  a period  not  to  exceed  the  life  of 
this  Directive. 

B.  DoD  Components  will  review  their  existing  regulations,  speci- 
fications, and  standards  modifying,  cancelling,  or  supple- 
menting them  as  required  to  ensure  consistency  with  the  policy 
in  this  Directive. 

C.  D6D  Components  will  develop  and  implement  a disciplined 
approach  to  the  management  of  software  design,  engineering, 
and  programming  which  will  ensure  the  provision  of  effective 
software  at  minimum  life  cycle  cost.  To  assist  in  the 
achievement  of  this  objective,  DoD  Components  will,  as  a 
minimum: 

1.  Prepare  and  maintain  appropriate  guidance  documents  (e.g., 
guidelines,  checklists,  handbooks,  and  descriptive 
examples)  covering  requirements  definition,  development, 
acquisition,  operation,  and  support  issues  attendant  to 
computer  software  in  Defense  systems.  These  documents 
should  be  available  for  use  as  necessary  by  program 
managers  and  their  staffs  as  well  as  organizations  tasked 
with  specific  responsibility  for  developing,  acquiring, 
operating,  and  supporting  the  computer  reccxirce  elements. 

2.  Establish  and/or  maintain  appropriate  education,  training, 
and  experience  career  paths  with  accompanying  career  in- 
centives to  foster  the  development  and  retention  of  pro- 
fessional computer  resource  engineers,  managers,  and 
technicians . 

3.  Plan  and  execute  a coordinated  research  and  development 
program  to  identify  and  supply  the  technological  base 
needed  to  support  the  policy,  practice,  and  procedure  re- 
requirements of  this  Directive.  This  coordination  will 
be  accomplished  using  the  Technology  Coordinating  Paper 
(reference  (k)). 

VII.  EFFECTIVE  DATE  AND  IMPLEMENTATION 

This  Directive  is  effective  imoediately.  Five  copies  of  the 
Implementation  plan  shall  be  forwarded  to  the  Assistant  Secre- 
tary of  Defense  (Installations  and  Logistics)  for  approval. 
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CHARIER  (F 


DCC  MANAGEMENT  STEERING  COMMITTEE 
F®  ' 

EMBEDDED  COMPUTER  RESOURCES 


I.  BACKGROUND 

Current,  annual  expenditures  by  the  Department  of  Defense  on  the 
design,  development,  acquisition,  management  and  operation  sup- 
port of  computer  resources  embedded  within  and  integral  to 
weapons,  communications,  command  and  control,  and  intelligence 
systems  are  measured  in  the  billions  of  dollars.  At  the  6ame 
time  such  computer  resources  have  often  presented  critical  cost 
and  schedule  problems  during  the  development  and  acquisition  of 
new  defense  systems.  Even  after  system  implementation  and  field- 
ing the  software  has  often  proven  unreliable . To  correct  these 
problems  and  to  improve  tht  management'  of  embedded  computer 
resources  in  general,  a DoD  management  steering  committee  is 
hereby  formed.  This  committee  will  be  responsible  for  imple- 
menting this  Directive  and  will  operate  under  the  provisions  of 
this  Charter. 

II.  SCOPE 

A.  The  Management  Steering  Committee  for  Einbedded  Computer 
Resources  (MSC-ECR)A'  shall  implement  the  provisions  of  this 
Directive  and  issue  ensuing  policies  related  to  computer 
resources  which  are  embedded  within  major  Defense  weapon, 
command,  control,  communications,  and  intelligence  systems. 

B.  The  M3C-ECR  activities  will  not  encompass  the  field  of 
general  purpose,  commercially  available  Automatic  Data 
Processing  Equipment  (AD PE)  as  defined  and  administered  by 
references  (a),  (b),  (c),  and  (d)  of  this  Charter.  Working 
level  interfaces  will  be  maintained  with  the  ADFE  Community, 
however,  to  ensure  maximum  transferability  of  ideas  and  cross- 
utilization of  products. 

III.  OBJECTIVES 

The  objectives  of  the  MSC-ECR  are  fcxirfold: 

A.  Improve  the  management  of  computer  resources  embedded  in 
major  Defense  systems. 

Formerly  named  "Weapon  Systems  Software  Management  Steering 
Comal  ttee." 
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B.  Increase  visibility  of  canputer  resources  In  overall  system 
acquisitions , 

C.  Formulate  a coordinated  BoD  Technology  Base  Program  for  soft- 
ware basic  research,  exploratory  development,  advanced  devel- 
opment, and  technology  demonstrations  addressing  critical 
software  issues  that  can  be  recommended  to  the  Director, 
Defense  Research  and  Engineering. 

D.  Guide  the  assimilation  and  integration  of  computer  resource 
policy,  practice,  procedure,  and  technology  into  the  normal 
process  of  major  Defense  systems  acquisition. 

IV.  ACTIVITIES 


In  carrying  out  the  objectives  of  section  HI . , the  MSC-ECR 

shall: 

A.  Develop  proposed  future  policies,  or  changes  to  existing  pol- 
icies as  may  be  necessary  for  the  acquisition  and  management 
of  embedded  computer  resources  in  major  Defense  systems,  and 
oversee  the  implementation  of  policies  6tated  in  this 
Directive . 

B.  Advise  the  Principals  of  the  Defense  System  Acquisition  Review 
Council  on  general  policy  matters  and  on  specific  embedded 
computer  resource  issues  related  to  major  Defense  Systems. 

C.  Provide  recommendations  and  advice  to  DDR&E  on  Computer 
resource  R&D  technology  programs. 

D.  Provide  a focal  point  for  inter-  and  intra-Service  coordina- 
tion on  policy  and  management  issues. 

E.  Coordinate  technology  efforts  among  DoD  Components. 

F.  Review  DoD  Component  activities  for  compliance  with  the  pro- 
visions of  thi6  Directive. 

V.  CFOANIZATICK  & COMPOSITION 

The  K3C-ECR  shall  be  composed  of  an  Executive  Board  and  a Manage- 
ment Advisory  Board,  assisted  as  necessary  by  technical  panels 

working  in  areas  of  specialized  expertise. 

A.  The  Executive  Board  shall  consist  of  one  designated  repre- 
sentative from  Assistant  Secretary  of  Defense(lnstallations 
and  Logistics),  DDR&E,  Director,  Telecommunications  and 
Command  and  Control  Systems,  Assistant  Secretary  of  Defense 
(Comptroller),  and  Assistant  Secretary  of  Defense (Intelli- 
gence). The  Executive  Board  will  be  chaired  by  ASD(l&L). 

All  decision-making  power  of  the  M5C-BCR  shall  be  vested 
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in  the  Executive  Board;  opinions  and  decisions  of  the  Board 
vi 11  be  expressed  by  the  chairman,  acting  *6  principal  spokes- 
man for  the  ICC-ECR,  and  vill  be  based  on  concurrence  of  all 
Board  members.  If  concurrence  cannot  be  achieved,  the 
divergent  views  will  be  forwarded  with  majority  and  minority 
reports  for  resolution  by  OSD  staff  principals  or  the  Deputy 
Secretary  of  Defense. 

B.  The  Management  Advisory  Board  shall  consist  of  representatives 
of  DoD  Components  as  folltws: 


Army 

Navy 

Air  Force 

Office  of  the  Joint  Chiefs  of  Staff 

Defense  Communications  Agency 

National  Security  Agency 

Defense  Advanced  Research  Projects  Agency 

TRI-TAC 

Deputy  Director  (Test  and  Evaluation ), 
CDDR&E 


3 members 
3 members 
3 members 

1 member 

2 members 
2 members 
2 members 
1 member 

1 member 


RESPONSIBILITIES 


Responsibilities  pursuant  to  the  provision  of  this  Charter  and  of 
this  Directive  shall  be  as  follo/s: 

A.  The  Executive  Board  of  the  Management  Steering  Committee  shall: 


1.  Develop  policy,  or  changes  to  existing  policy  as  may  be 
necessary  for  the  acquisition  and  management  of  computer 
resources  in  major  Defense  systems,  and  oversee  their 
accomplishment  .' 

2.  Advise  the  Principals  of  the  Defense  Systems  Acquisition 
Review  Council  on  general  policy  matters  and  on  specific 
computer  resource  issues  applicable  to  DSARC-managed 
programs. 

3.  Provide  recommendations  and  advice  to  the  Director  Defense 
Research  and  Engineering  on  computer  resource  R&D  tech- 
nology programs. 

Review  DoD  Component  activities  for  compliance  with  the 
provisions  of  tbi6  Directive. 

5.  Assist  the  Chairman  of  the  OSD  Cost  Analysis  Improvement 
Group  (CAIG)  in  preparing  Independent  cost  estimates  for 
major  Defense  Systems. 
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B.  The  Management  Advisory  Board  of  the  Management  Steering 
Committee  shall,  for  major  Defense  Systems: 

1.  Conduct  policy  Impact  assessments  and  analyses  for  the 
Executive  Board  in  both  technical  and  managerial  ateas 
relating  to  computer  resources. 

2.  Serve  as  focal  points  for  inter-  and  Intra-Service  coor- 
dination on  policy  and  management  Issues. 

3.  Coordinate  technology  efforts  among  DoD  Components. 

4.  Review  computer  resource  technology  programs  for  policy 
consistency,  relevancy  and  impact;  advise  Executive 
Board  of  meaningful  technology  findings,  results,  and 
product  developments. 

5.  Publicize  appropriate  management  and  technological 
developments  related  to  computer  resources,  throughout 
DoD  and  industry. 

C.  The  Management  Advisory  Board  will  assist  the  Executive 
Board  in  fulfulling  the  objectives  of  the  MSC— ECR,  and  the 
members  will  act  as  focal  points  for  their  respective  DoD 
Components  in  the  areas  of  embedded  computer  resources. 

VII.  TECHNICAL  PANELS 


Adhoc  Technical  Panels  may  be  formed  at  the  direction  of  the 
MSC-ECR  to  examine  problems  requiring  specialized  and  detailed 
''  expertise.  Any  panel  so  formed  will  be  governed  by  its  own 
Charter,  which  must  be  approved  by  the  Executive  Board,  and 
will  report  to  the  membership  of  the  16C-ECR.  An  appropriate 
Chairman  of  the  Executive  Board.  Membership  on  the  panel  will 
be  determined  by  the  Panel  Chairman,  and  may  be  drawn  from  the 
DoD  Components  or  from  industry  as  appropriate  for  the  task  at 
hand. 

VIII.  METHOD  OF  OPERATION 

A.  The  MSC-ECR  shall  meet  quarterly,  or  upon  the  call  of  the 
Chairman.  The  agenda  trill  be  set  by  the  presiding  Chairman, 
with  concurrence  by  members  of  the  Executive  Board. 

B.  The  ASD(I6L) , or  hi6  designated  representative  shall  act  a6 
Executive  Secretary  to  the  MSC-ECR,  and  shaj.1  be  responsi- 
ble for  preparing  the  minutes  and  administering  the  overall 
affairs  of  the  committee.  Minutes  of  all  meetings  shall  be 
distributed  no  later  than  30  calendar  days  after  the  subject 
meeting  is  adjourned.  The  ASD(I&L)  shall  provide  adminis- 
trative support  to  the  MSC-ECR. 
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IX.  DEFINITIONS 


The  terns  defined  In  the  basic  Directive  shall  be  applicable  to 
this  Charter  and  the  functioning  of  the  MSC-ECR. 

X.  REFERENCES 


A.  Office  Management  Budget  Circular  A-71,  "Responsibilities  for 
the  Administration  and  Management  of  Automatic  Data  Processing 
Activities,"  March  6,  1965 

B.  DoD  Directive  5100.40,  "Responsibilities  for  the  Administration 
of  the  DoD  Automatic  Data  Processing  Program,"  August  19,  1975 

C.  DoD  Directive  4105.55,  "Selection  and  Acquisition  of  Automatic 
Data  Processing  Resources,"  April  5,  1973 

D.  DoD  Directive  4160.19,  "Department  of  Defense  Automatic  Data 
Processing  Equipment  Reutilization  Program,"  April  5,  1973 
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$9  MAY  1974 


MEMORANDUM  FOR  Assistant  Secretaries  of  the  Military  Departments  (R&D 

Director,  Defense  Advanced  Research  Projects  Agency 
Director,  Defense  Nuclear  Agency 
Director,  Defense  Intelligence  Agency 

SUBJECT:  Technology  Coordinating  Papers 


The  concept  of  Technology  Coordinating  Papers  has  been  evolving  for  more 
than  four  years.  In  this  period,  essentially  one  of  trial  and  error,  the 
concept  has  been  clarified  and  certain  problems  associated  with  the  overall 
implementation  have  been  surfaced.-  As  a result,  we  are  now  in  a better 
position  to  restate  the  general  requirements  for  TCP's,  their  utilization, 
their  content,  the  management  of  the  TCP  process,  the  review  and  critique 
process,  and  the  distribution  of  the  coordinated  documents.  This  memo- 
randum provides  the  overall  guidance  to  DoD  personnel  involved  in 
preparation  or  revision  of  those  TCP's  either  in  process  or  planned  for 
the  future,  and  supersedes  prior  memoranda  of  19  January  1972,  18  August 
1972,  and  29  March  1973  on  this  subject. 

GENERAL  REQUIREMENTS  FOR  TCP's 

The  TCP's  which  have  been  published  are  proving  invaluable  to  R&  D 
managers  as  the  best  means  to  provide  a bounded  overview  of  selected 
segments  of  the  DoD  technology  base.  TCP's  have  served  to  answer  the 
following  questions: 

• Are  there  needless  overlaps  and  duplications? 

• Are  there  vital  defense  research  areas  which  are  underfunded 
and  even  missing  from  the  base? 

• Are  sufficient  coordination  and  interchange  taking  place  among 
the  Services  to  maximize  the  return  from  resources  being 
applied  to  a given  area? 
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• Are  the  priorities  set  correctly;  that  is,  are  the  spending  levels 
for  the  various  areas  consistent  with  the  requirements  in  those 

areas? 

• Are  future  weapon  system  requirements  being  acknowledged  in 
the  more  applied  work  being  conducted  within  the  technology 
base  ? 

• How  does  the  overall  program  match  priorities  and  mission  area 
deficiencies  ? 

These  are  the  kinds  of  questions  which  are  of  concern  to  both  the  DDRbE 
and  the  Assistant  Secretaries  (RbD)  & Military  RbD  Chiefs  of  the  Military 
Departments  who  are  responsible  for  overseeing  the  Service  programs. 

In  addition,  the  Secretary  of  Defense  and  the  Congress  have  questions 
concerning  the  relative  value  of  the  diverse  activities  contained  in  the 
technology  base  --  questions  which  can  best  be  answered  by  showing 
how  the  various  pieces  fit  together  to  make  a coherent  whole.  The 
TCP's  have  also  served  as  a device  for  improving  interservice  and 
Defense  Agency  communications  in  most  technology  areas.  In  tome 
areas,  information  from  the  TCP'6  has  also  provided  the  basis  for  the 
dissemination  of  information  on  technology  programs  and  future  needs 
to  the  industrial  and  academic  sectors.  For  these  reasons,  we  shall 
continue  to  prepare  TCP's  in  the  following  technology  base  program 
areas: 

• Propulsion  Technology,  Missiles  and  Space  Vehicles 
* t*  Medical  and  Biological  Sciences 

• Materials  Technology 

• Structures  Technology 

• Aircraft  Propulsion  Technology 

• Aeronautical  Vehicle  Technology 

• Human  Resources  Technology 

• Environmental  Sciences 

• Electronic  Devices 

• Weapons  Technology 
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• Surface  Vehicles 

• Electronics  Technology  (to  be  initiated  in  FY  1975) 

UTILIZATION  OF  TCP’s 

While  TCP's  have  proved  an  effective  mechanism  for  spotting  duplicative, 
underfunded,  or  missing  programs,  they  have  not  generally  fulfilled  one 
of  the  originally  intended  roles;  that  of  forming  a basis  for  organizing 
work  in  specific  segments  of  the  technology  base  where  appreciable 
multi-Service  activity  or  interest  exists.  Neither  have  they  been  optimally 
utilized  by  all  levels  of  Service  R&D  Management  as  an  aid  in  making 
decisions  on  prudent  allocation  of  resources  in  the  various  technology  areas 
Only  in  some  cases  have  TCP's  been  used  as  data  bases  for  the  general 
planning  process  at  the  Service  staff  and  systems  command  levels.  In 
short,  it  does  not  appear  that  middle  management  in  the  Services  has  taken 
full  advantage  of  the  information  contained  in  the  TCP  documents  This 
deficiency  could  be  corrected  if  the  TCP's  were  made  a part  of  the  basic 
documentation  for  use  at  all  management  levels  in  the  preparation  of  budget 
and  apportionment  plans.  Additionally,  the  utilization  of  TCP  information 
in  the  preparation  of  overall  investment  strategy  analyses  for  specific 
technology  areas  would  be  a valuable  adjunct  to  the  planning  documents  of 
the  individual  Services. 

CONTENT  OF  TCP's 


The  use  of  a standard  format  or  a standard  table  of  contents  for  a TCP 
is  not  required.  The  format  should  be  the  prerogative  and  responsibility 
of  the  Service/Agency  team  preparing  the  TCP.  However,  if  the  objectives 
of  the  TCP  process  are  to  be  achieved,  these  documents  should  contain  at 
least  the  following  information; 

• An  examination  of  the  impact  of  both  near  term  and  future 
military  requirements  by  mission  area  as  they  might 
influence  that  technology 

• A description  of  the  current  and  future  DoD  program  in  that 
technology  area  and  the  degree  to  which  the  program  satisfies 
firm  military  requirements.  This  should  include  a summary 
of  work  content,  designation  of  the  sponsoring  Service  or 
Defense  Agency  for  major  tasks,  and  where  these  tasks  are 
being  performed  (in-house  or  major  contractors,  by  name). 
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Within  this  description  an  explicit  discussion  of  the  motivations 
and  relevancy  of  the  6.  1 Research  projects  should  be  given. 

The  TCP  should  also  highlight  the  non-system  6.  3 Advanced 
Technology  Demonstration  projects  with  appropriate  discussions 
as  to  their  potential  payoff  in  terms  of  improved  operational 
capability,  cost  reduction,  or  cost  avoidance. 

• A summary  table  matrix  which  correlates  technical  sub-areas 

with  sponsor.  Program  Element,  and  Project  number.  Because 
of  the  problems  associated  with  the  distribution  of  outyear  fiscal 
data,  TCP's  will  be  written  to  contain  financial  data  for  only  the 
current  and  budget  years.  Data  from  preceding  fiscal  years 
should  be  selectively  included  to  indicate  significant  trends. 
Complete  FYDP  data  will  not  be  shown  in  the  TCP,  although 
anticipated  trends  in  funding  levels  may  be  indicated  either 
quantitatively  or  qualitatively.  This  restriction  is  not  meant 
to  exclude  the  consideration  of  FYDP  data  in  the  preparation 
of  a TCP.  which  can  be  very  useful,  or  to  inhibit  publication 
of  FYDP  data  in  an  appendix  or  supplement  to  the  TCP. 

• Identification  and  description  (if  available)  of  other  DoD 
programs  (i.  e.  , Manufacturing  Technology,  Component 
Improvement,  etc.)  non-DoD  programs  (NASA,  AEC,  NSF, 
etc.)  or  other  major  efforts  (IRbD),  if  any,  which  have  a 
significant  impact  on  the  technology  area,  and  an  assessment 
of  that  impact. 

\ A short  assessment  of  the  technology  area  itself,  including 
mission  area  deficiencies,  a brief  recount  of  significant 
historical  trends  and  expected  future  trends,  significant 
recent  accomplishments  and  (where  instructive)  significant 
recent  negative  results. 

No  restrictions  on  the  size  of  TCP's  can  reasonably  be  imposed.  Some 
TCP’s  will  be  comprised  of  a single  document  of  perhaps  50-80  pages 
in  length  whereas  others  will  be  as  long  as  several  hundred  pages  because 
of  the  diversity  of  that  technology  area.  All,  however,  should  contain 
an  Executive  Summary  (maximum  of  20-25  pages)  of  the  salient  information 

in  the  document. 

MANAGEMENT  OF  TCP  PROCESS 

• The  DD(R&AT)  is  responsible  for  the  overall  implementation 
of  the  TCP  process. 
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• The  DD(RbAT)  is  responsible  for  ensuring  that  each  basic  TCP 
is  as  concise  as  possible  and  that  all  background  information 

is  prepared  in  a useful  format. 

• The  DD(RitAT)  is  responsible  for  determining  the  rate  and 
frequency  of  preparation  of  each  TCP.  Annual  revisions  of 
TCP's  will,  therefore,  not  be  automatic,  but  an  annual  review 
of  each  TCP  will  be  made  to  determine  whether  to  amend, 
update,  rewrite,  supplement  or  make  no  change. 


• It  is  not  required  that  working  drafts  or  for-comment  drafts 
of  TCP's  be  thoroughly  staffed  or  coordinated  at  the  middle 
or  upper  management  levels.  Forty-five  days  will  be  allowed 
for  review  of  the  coordination  draft  of  each  TCP.  The  Assistant 
Secretaries  (R&D)  of  the  Military  Departments  may,  at  their 
discretion,  delegate  coordination  authority.  I have  delegated 
the  DDR&E  coordination  authority  to  the  DD(R&AT). 


DISTRIBUTION  OF  TCP's 

• The  basic  TCP  documents  will  not  be  distributed  to  industry. 

They  will  be  selectively  distributed  to  other  Federal  Agencies 
such  as  NASA,  CIA,  and  NSF  and  to  the  Congress  as  appropriate. 
The  DD(R&AT)  will  determine  the  appropriateness  of  TCP 
distribution  outside  of  DoD. 


• Initial  and  secondary  distribution  of  coordination  TCPs  will 
be  made  through  the  Defense  Documentation  Center  (DDC). 
Distribution  to  other  than  in-house  organizations  will  require 
the  express  approval  of  DD(R&AT). 

• Every  effort  will  be  made  to  distribute  TCP  information  (not 
TCP's)  concerning  technology  requirements  to  industry, 
academic  and  other  non-government  personnel.  The  appropriate- 
ness of  these  distributions  will  be  determined  by  DD(R&AT). 

• Draft  and  coordinated  TCP's  will  be  distributed  to  appropriate 
Service  laboratories.  The  appropriateness  of  these  distributions 
will  be  determined  by  the  Military  Departments. 


5000.  29  (Enel  3) 
Apr  26,  76 


REVIEWS  AND  CRITIQUES  OF  TCPjS 

• It  shall  be  the  reaponeibllity  of  the  DD(RfcAT)  to  obtain  critique# 
cf  TCP'a  as  appropriate.  The  nature  of  theae  reviews  will  vary 
with  the  scope  and  content  of  the  individual  TCP.  The  reviewers 
may  be  comprised  of  professional  staff  from  federal  Contract 
Research  Centers  (FCRC'a),  members  of  quasi- government 
institutions  such  as  the  National  Academy  of  Sciences  (NAS), 

or  selected  personnel  from  private  industry  or  academia. 

• When  members  of  private  industry  or  academia  are  utilized  to 
review  TCP's,  they  will  conduct  their  review  in  the  Pentagon 

and  will  not  be  permitted  to  take  the  documents  from  the  building.  - 

• The  results  of  all  such  reviews  will  be  passed  to  the  Military 
Departments  and  appropriate  Defense  agencies  for  information 
and  comment  if  appropriate. 
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(j)  Armed  Services  Procurement  Regulation,  Section  IX,  Parts  5 and  6 

(k)  "Interim  List  of  DoD  Approved  High  Order  Languages,"  (To  be  pub- 

lished) 

(l)  Director,  Defense  Research  and  Engineering  Memorandum,  "Technology 

Coordinating  Papers,"  May  29,  1974  (enclosure  3) 

(m)  Defense  Standardization  Manual  4120. 3M,  "Standardization  Policies, 
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DATE  November  24,  197 


ASD(I6L)/DDR6E/DTACCS/ASD(C) 

Department  of  Defense  Instruction 


SUBJECT  : Interim  List  of  DoD  Approved  High  Order  Programming  Languages  ( 

References:  (a)  DoD  Directive  5000.29,  "Management  of  Computer  Reaources 

in  Major  Defense  Systems,"  April  26,  1976 

(b)  DoD  Directive  5000.1,  "Acquisition  of  Major  Defense 

Systems,"  December  22,  1975 

(c)  DoD  Directive  5100.40,  "Responsibility  for  the  Adminis- 

tration of  the  DoD  Automatic  Data  Processing  Program," 
August  19,  1975 

I.  PURPOSE 

This  Instruction  specifies  the  High  Order  Programming  Languages  (H0L) 

which  are  approved  for  use  in  conjunction  with  reference  (a). 

II.  APPLICABILITY  AND  SCOPE 

A.  The  provisions  of  this  Instruction  apply  to  the  Office  of  the 
Secretary  of  Defense,  the  Military  Departments,  the  Organization 
of  the  Joint  Chiefs  of  Staff,  and  the  Defense  Agencies  (herein- 
after referred  to  collectively  as  "DoD  Components"). 

B.  Its  previsions  encompass  the  selection  of  HOL  for  the  develop- 
ment of  software  in  major  programs  of  defense  systems  acquisition 
as  designated  by  the  Secretary  of  Defense  (described  in  section  II 
of  reference  (b))>  as  well  as  for  defense  system  acquisitions  that 
do  not  fall  in  the  "major  acquisition"  category. 


C.  Excluded  from  the  provisions  of  this  Instruction  are: 


1.  Commercially  available  software  for  use  with  automatic  data 
processing  assets  as  defined  and  administered  under  reference 
(c). 

2.  Those  application  or  user  oriented  languages  which  do  not  fall 
within  the  category  of  a programming  language  (e.g..  User 
Requirements  Languages,  Automatic  Test  Equipment  Languages, 
Production  Control  Languages,  simulation  Languages,  and  Analyst 
Aid  Languages). 

D.  The  provisions  of  the  Instruction  are  not  to  be  applied  retro- 
actively on  any  defense  systems  where  a language  commitment  has 
already  been  made,  nor  is  it  to  be  Interpreted  as  prejudicial  to 
language  selections  occurring  before  DoD  policy  formulation. 


III.  DEFINITION 


For  purposes  of  this  Instruction  a HOL  is  one  which  provides  com- 
pression of  computer  instructions  auch  that  one  HOL  statement 
represents  many  machine  language  instructions.  It  is  non-problem- 
specific  and  is  used  by  programmers  to  communicate  with  a computer. 

IV.  POLICY 

A.  General 

1.  This  Instruction  and  DoD  Directive  5000.29  (reference  (a)) 
is  designed  to  reduce  the  proliferation  of  HOL  in  defense 
systems  and  to  ensure  control  of  those  HOLs  which  are 
approved . 

2.  DoD  approved  HOLs  will  be  used  to  develop  defense  system 
software,  unless  it  is  demonstrated  that  none  of  the 
approved  HOLs  are  cost  effective  or  technically  practical 
over  the  system  life  cycle  (reference  (a),  subsection 

V.G.).  Each  DoD  Component  will  designate  in  its  instruc- 
tion implementing  DoD  Directive  5000.29  (reference  (a)) 
one  office  authorized  to  approve  requests  for  such  excep- 
tions. The  designated  approval  authority  will  maintain 
appropriate  records  to  support  periodic  review  by  the 
Management  Steering  Committee  for  Embedded  Computer 
Resources . 

3.  Each  DoD  approved  HOL  will  be  assigned  to  a designated 
control  agent  who  will  be  responsible  for  such  activities 
as  assuring  language  stability  and  configuration  manage- 
ment, validating  compliance  of  compiler  implementations 
with  the  standard  language  specifications,  gathering  data 
as  to  the  use  of  the  languages,  and  for  disseminating 
information,  compilers,  and  tools  (reference  (a)  subsec- 
tion V.G.) . 

B.  Approved  High  Order  Programming  Languages 

1.  The  DoD  approved  High  Order  Programming  Languages,  and 
their  defining  specification  documents  are: 

a.  CMS- 2 "CMS-2Y  Programmers  Reference  Manual," 

M-5049,  FDCSSA , San  Diego.  CA., 

October  1,  1976:  and  "CMS-2M  Computer 
Program  Performance  Specifications," 

NAVELEX  0967LP-598-2210. 
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b.  SPL-1  "SPL-l  Language  Reference  Manual,"  Intermetrics 

Report  No.  172-1. 

c.  TACPOL  CPCEI  Part  1 Specification  EL-CG-00043082C 

Volume  1,  April  16,  1971  with  ECO  Modifica- 
tions (Appendix  10). 

d.  JOVIAL  Military  Standard  (MIL-STD)  1588  (USAF)  for 

J3  and  MIL-STD-1589  (USAF)  for  J73. 

e.  COBOL  ANSI  X3.23  - 1974. 

f.  FORTRAN  ANSI  X3.9  - 1974. 

2.  The  languages  CMS-2  and  SPL-1  shall  be  controlled  within 
DoD  by  the  Department  of  the  Navy. 

3.  The  language  TACPOL  shall  be  controlled  by  the  Department 
of  the  Army. 

4.  The  language  JOVIAL  shall  be  controlled  by  the  Department  . 
of  the  Air  Force. 

5.  The  languages  COBOL  and  FORTRAN  shall  be  controlled  by  the 
Office  of  the  Assistant  Secretary  of  Defense  (Comptroller) 
acting  with  the  National  Bureau  of  Standards  and  the 
American  National  Standards  Institute  (DoD  Directive  5100.40, 
reference  (c)). 

V.  RESPONSIBILITIES 

A.  The  Management  Steering  Committee  for  Embedded  Computer 
Resources,  DoD  Directive  5000.29  (reference  (a)),  shall  oversee 
and  coordinate  the  accomplishment  of  the  policies  in  this 
Instruction,  and  advise  the  principal  assistants  of  the  Office 
of  the  Secretary  of  Defense  on  matters  related  to  this  policy. 

B.  The  Military  Departments  will  designate  control  agents  for 
each  H0L  under  their  purview. 

C.  The  HOL  control  agents  ao  designated  by  the  Military  Depart- 
ments are  authorized  to  update  their  designated  language  with 
compatible  extensions  and  improvements  to  satisfy  validated 
requirements.  Such  extensions  (e.g.  new  syntax  and/or  new 
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semantics)  should  not  be  made  more  often  than  once  per  year. 
The  COBOL  and  FORTRAN  control  agents  must  comply  with  the  cur- 
rent approved  version  of  the  American  National  Standards 
Institute. 

VI.  EFFECTIVE  DATE 

This  Instruction  is  effective  immediately. 


fsistant  Secrete 
(Installations  an? 


of  Defense 
Logistics) 


Assistant  Secret 
( 


of  Defense 


Director  Telecommunications  and 
Command  and  Control  Systems 


Director  of  Defense  Research  and 
Engineering 
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APPENDIX  B 

CONSULTATION  AND  ASSISTANCE 
CONTACT  LIST 


MSC-ECR  MEMBERSHIP  ROSTER 


9-21-77 


MANAGEMENT  ADVISORY  BOARD  (CONTINUED) 

Mr.  James  Campbell  Col  Robert  Zlemlckl 

ASN(R*D)  AF/RDPV 

$E813,  Pentagon  4D239,  Pentaoon 

Phone:  695.3623  Phone:  695-0156 

RADM  Crawford  A.  Easterling  Col  Robert  J.  Yunk 

Command,  Control  ( Information  Systems  Af/LGYE 

Division,  OP-9A2,  U.S.  Navy 
5E569*  Pentagon 


Phone : 695-6667 

Capt  E.  J.  Rieher 
Chief  of  Naval  Material 
(MAT  09Y),  Crystal  Plaza  #6 
Room  681*,  Deoartment  of  the  Navy 
Washington,  D.C.  20360 
Phone:  692-8615 

Col  R.  0.  Kitts 

Headquarters  U.S,  Marine  Corps 
CMC(CC)  Navy  Annex  3018 
Systems  Division 
(Guard  Mai  1 Please) 

Phone:  69^-1 162 


Dr.  Walter  R.  Beam 
SAFRD 

4D961,  Pentagon 
Phone  697-3311 

Mr.  Leighton  Lomas 
AF/RDM 

603^3*  Pentaoon 

Phone:  697-6093 

Colonel  William  S.  E91 inton 
AF/X0A 

5D3**1,  Pentagon 
Phone  695-9204/697-1925 


4D260,  Pentagon 

Phone:  697-8159/695-0080/697-8135 

Director,  National  Security  Agency 
ATTN:  Mr.  Donald  M.  Rickerson-R251 
Fort  Meade,  MD  20755 
Phone:  688-7915 

Mr.  Paul  High 

Defense  Intelligence  Agency 
ATTN:  SP 

Washington,  D.C.  20301 
Phone:  697-0917 

LTC  William  A.  Whitaker 
Special  Assistant  to  the  Director 
Defense  Advanced  Research  Project 
Agency  (DARPA) 

U00  Wilson  Boulevard 
Arlington,  VA  22209 
Phone:  694-1 1 39 

Capt  Harold  Adams 
TRI-TAC  TT-E-ED 
Fort  Monmouth,  NJ  07703 
Phone:  Autovon  8-992-8292 

Col  D.  W.  Lambrecht 
TRI-TAC 

Fort  Monmouth,  NJ  07703 
Phone:  Autovon  8-992-8355 


